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ABSTRACT 


The  terminal  environment  in  the  aviation  domain  is  referred  to  as  the  region 
within  close  proximity  of  airports.  Performance  in  this  environment  leaves  little  room  for 
error  and  requires  that  pilots  are  fully  aware  of  their  situation  and  surroundings.  This  is 
especially  true  when  conducting  an  instrument  approach  (IA).  Currently,  United  States 
Marine  Corps  (USMC)  KC-130J  units  do  an  exceptional  job  of  training  their  pilots  in  this 
environment.  However,  the  emergence  of  new  technologies  affords  the  opportunity  to 
address  shortfalls  of  legacy  training  approaches.  To  this  end,  this  thesis  uses  a  quasi- 
instructional  systems  design  approach  to  develop  a  prototype  IA  part-task  trainer  (PTT) 
using  commercial  off-the  shelf  (COTS)  systems  to  enhance  current  training  methods  to 
allow  for  training  opportunities  that  are  not  limited  by  schedules  or  location.  After 
the  development  of  the  PTT,  subject-matter  experts  evaluated  the  prototype  training 
system.  Their  feedback  suggests  that  the  system  reliably  re-created  necessary  cues 
(cockpit  displays,  environmental  conditions,  communications,  time,  and  space)  to  build 
on  users’  existing  knowledge  and  support  situation  awareness  training  for  IA. 
Ultimately,  the  proof-of-concept  prototype  created  for  this  thesis  is  envisioned  to  be 
part  of  a  family  of  solutions  capable  of  addressing  existing  training  gaps  in  a  cost- 
effective  manner. 
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I.  INTRODUCTION 

A.  RESEARCH  DOMAIN 

An  instrument  approach  (IA)  is  one  method  used  to  successfully  land  an  aircraft 
at  an  airport.  When  an  IA  is  being  flown,  there  are  several  basic  procedures  that  the  pilot 
flying  (PF)  must  accomplish  in  order  to  successfully  land  that  aircraft.  These  basic 
procedures  typically  follow  a  sequence  that  does  not  change  from  approach  to  approach, 
so  the  aviator  becomes  very  good  at  accomplishing  this  sequence  successfully.  However, 
when  an  emergency  situation  occurs  during  the  flight  and  an  IA  must  be  flown  to  land  the 
aircraft,  these  basic  procedures  could  become  difficult  to  accomplish.  These  emergency 
situations  often  require  heighten  situation  awareness  (SA)  so  that  the  basic  procedures 
can  be  accomplished  while  also  successfully  completing  the  procedures  of  the 
emergency.  Having  sound  SA  is  crucial  in  these  situations  for  one  major  reason:  if  all 
basic  and  emergency  procedures  are  not  successfully  completed  during  the  IA,  a 
catastrophic  event  could  occur. 

United  States  Marine  Corps  (USMC)  KC-130J  replacement  pilots  (RP)  receive 
initial  IA  training  at  a  Fleet  Replacement  Detachment  (FRD).  At  the  FRD,  a  RP  conducts 
ground  school,  which  consists  of  self-paced  computer-based  training  (CBT)  and 
instructor-led  discussions  focused  on  studying  the  missions  of  the  KC-130J.  In  addition, 
there  is  material  within  different  publications  that  a  RP  needs  to  know.  The  instrument 
approach  procedure  (IAP)  elements  of  ground  school  consist  of  generic  IAP  for  all 
aviators  and  IAP  specific  to  KC-130J  aviators.  The  RP  will  then  begin  to  fly  IAs  in  the 
KC-130J  simulator.  Initially,  these  IAs  consist  of  honing  these  basic  procedures  so  that 
the  RP  could  acquire  a  solid  level  of  confidence  while  executing  those  procedures.  In 
these  early  stages  of  training  at  the  FRD,  it  is  commonly  accepted  that  it  is  more 
important  that  the  RP  polishes  these  basic  procedures  rather  than  successfully  fly  the 
simulator.  Successful  flight  of  the  simulator  will  come  with  practice,  and  it  will  be 
enhanced  with  mastery  of  the  basic  IAPs. 
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B.  RESEARCH  PROBLEM  AND  MOTIVATION 


When  emergency  procedures  (EP)  are  introduced  to  an  IA,  the  aviators,  especially 
RPs,  could  experience  some  difficulties.  Repetition  of  basic  procedures  is  extremely 
important  to  a  RP;  the  repetitive  training  approach  can  build  the  required  foundation  of 
these  procedures  so  that  they  become  second  nature.  However,  aviators  cannot  use  the 
simulator  and  aircraft  to  groom  and  acquire  these  procedures  at  their  leisure.  The 
simulator  and  aircraft  follow  a  very  rigid  schedule,  and  the  aviator  may  only  have  access 
to  the  simulator  and  aircraft  two  times  a  week.  Whereas  the  simulator  and  aircraft  provide 
potentially  better  opportunity  to  properly  practice  the  procedures,  their  availability  does 
not  allow  for  the  consistent  repetition  that  is  necessary. 

Another  technique  called  chair  flying  is  a  non-computer  based  technique,  and 
aviators  use  it  to  practice  any  procedure  on  their  own;  this  is  typically  done  without  the 
supervision  of  instructors.  This  approach  has  modest  demand  on  resources — chair  flying 
does  not  require  anything  but  a  chair.  Though,  chair  flying  is  only  useful  if  the  aviator  is 
practicing  all  procedures  properly.  This  is  especially  true  of  a  RP  because  at  that  stage  in 
their  career  and  training  the  RP  may  not  have  the  ability  to  recognize  that  he  or  she  is 
practicing  the  procedure  incorrectly.  It  is  worth  emphasizing  that  negative  practice  could 
transfer  to  negative  execution  of  procedures  within  the  simulator  and  aircraft. 

The  instructor-led  discussions  are  the  only  portion  of  the  classroom  environment 
that  is  required  of  the  student.  The  knowledge  of  the  computer-based  training  and 
readings  are  upon  the  student.  However,  the  RP  is  required  to  combine  the  knowledge 
from  these  different  avenues  and  apply  it  mentally  and  physically  within  the  simulator 
and  aircraft.  The  act  of  combining  the  required  mental  knowledge  with  the  required 
physical  actions  of  operating  the  cockpit  while  flying  the  aircraft  can  be  overwhelming  to 
the  RP. 

Proper  repetitive  practice  of  any  procedure  is  essential  to  the  development  of  this 
procedure.  Since  simulators  and  aircraft  are  not  readily  available  and  chair  flying  does 
not  ensure  proper  practice,  the  development  of  a  PTT  could  address  that  gap.  If  such  a 
PTT  would  use  a  COTS  platform  that  is  inexpensive  and  easy  to  acquire,  such  as  a  tablet, 
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the  PTT  would  afford  the  aviator  the  cost-effective  opportunity  to  practice  procedures 
anywhere  at  any  time.  General  Robert  B.  Neller  (2016),  the  Commandant  of  the  Marine 
Corps  (CMC),  has  stated  that  the  Marine  Corps  can  be  “enabled  by  technology,  we  will 
increase  the  amount  of  training  each  unit  can  accomplish — to  ‘increase  the  reps’  in 
mentally  and  physically  stressing  environments  for  all  elements  of  the  MAGTF  before 
they  do  so  on  the  battlefield”  (p.  8).  With  the  guidance  of  the  CMC,  the  development  of 
such  a  PTT  becomes  imperative  to  the  Marine  Corps  future  training  programs.  This  type 
of  affordable  yet  highly  capable  devices  has  a  great  potential  of  bridging  a  gap  that 
currently  exists  in  the  training  pipeline  of  any  community  in  the  Marine  Corps. 

C.  RESEARCH  QUESTIONS 

The  foundation  of  the  thesis  is  based  on  the  following  research  questions: 

•  What  is  the  feasibility  of  developing  a  prototype  virtual  reality  (VR) 
system  that  supports  the  training  of  instrument  approach  procedures  and 
situation  awareness? 

•  Are  there  training  gaps  identified  in  connection  with  procedures  and 
situation  awareness  needed  to  conduct  an  instrument  approach  by  the 
aviators? 

•  What  cues  (cockpit,  environmental,  CRM,  ATC,  haptic,  etc.)  should  KC- 
130J  FRD  students  attend  to  in  support  of  maintaining  situation  awareness 
during  instrument  approaches? 

D.  SCOPE 

The  scope  of  this  thesis  is  to  investigate  current  techniques  used  by  Marine  Corps 
KC-130J  squadrons  and  the  FRDs  to  train  aviators  for  IAP.  A  PTT  will  be  developed  as  a 
proof-of-concept  prototype  with  a  goal  to  become  a  supplemental  tool  to  the  current 
training  that  aviators  receive.  A  significant  portion  of  this  work  will  address  the  aspects 
of  SA  and  how  SA  relates  to  IAP.  The  study  also  includes  gathering  of  informal  feedback 
during  demo  sessions.  A  training  effectiveness  study  is  not  part  of  this  thesis,  yet  it  is 
envisaged  as  elements  of  follow-up  work. 
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E.  APPROACH 


The  approach  used  in  support  of  the  thesis  research  has  consisted  of  analyzing 
literature  related  to  SA  and  how  a  PTT  enhances  SA  so  that  the  basic  IAP  are  committed 
to  memory  and  conducted  without  thought.  In  addition,  a  user  survey  is  conducted  to 
provide  a  better  understanding  of  preferences,  habits,  and  recommendations  from  the 
fleet.  The  research  of  SA  and  the  survey  results  were  then  used  to  design  and  develop  the 
PTT.  A  demo  of  the  prototype  system  was  given  to  fellow  aviators,  and  their  informal 
feedback  was  acquired  to  help  understand  the  value  of  the  developed  PTT. 

F.  METHODOLOGY 

1.  Introduction 

The  basis  for  this  thesis  is  the  work  presented  in  a  past  NPS  thesis  involving 
Marine  aviation  (Attig,  2016).  This  thesis  is  not  continuation  work  of  those  results  but 
rather  an  effort  to  apply  the  basic  concepts  presented  there  to  a  different  domain  of 
Marine  aviation — IAPs — and  provides  a  novel  PTT  that  could  become  a  valuable  tool  to 
the  Marine  Corps’  future  training  environment.  Initially,  multiple  missions  of  the  KC- 
130J  community  were  suggested,  and  after  much  deliberation,  the  IAPs  were  chosen 
because  they  are  among  the  first  missions  that  RPs  must  leam.  The  PTT  that  resulted 
from  this  thesis  effort  gives  the  aviator  the  ability  to  run  the  scenario  and  observe  the 
proper  procedures  associated  with  an  IAP  as  it  is  conducted  in  the  KC-130J. 

2.  Design  Process 

To  begin  with,  a  literature  review  of  situation  awareness,  knowledge  types,  and 
PTT  was  conducted.  These  issues  are  the  basis  of  this  thesis,  and  it  is  imperative  to 
express  how  the  information  acquired  about  each  topic  applies  to  the  idea  of  using 
technology  to  assist  with  future  training  aspirations  of  the  Marine  Corps.  The  task 
analysis  for  the  PTT  was  developed  from  the  KC-130J  Super  Hercules’  Naval  Air 
Training  and  Operating  Procedures  Standardization  (NATOPS)  documentation.  Within 
the  NATOPS,  sections  of  chapters  7  and  16  were  used  to  generate  the  elements  of  task 
analysis  (Chapter  7  was  focused  on  the  in-range,  approach,  and  before  landing  flows  and 


4 


checklists  material,  and  Chapter  16  addressed  the  standard  callouts  required  while 
executing  an  IA)  (Department  of  the  Navy,  2014).  In  order  to  assist  with  the  design  of  the 
PTT,  two  surveys  were  conducted  to  collect  valuable  information  from  current  IPs  and 
RPs.  Not  only  would  these  answers  provide  valued  information  for  this  thesis’  success 
but  the  information  was  also  used  to  inform  the  thoughts  for  follow-on  work.  Prior  to 
developing  these  surveys,  subject  matter  experts  were  questioned  to  assist  with  the 
development  of  the  survey  questions  from  the  current  atmosphere  across  the  KC-130J 
community. 

G.  THESIS  STRUCTURE 

The  remainder  of  this  thesis  is  structured  as  follows: 

•  Chapter  II  describes  SA,  COTS,  and  knowledge. 

•  Chapter  III  details  task  analysis  focused  on  instrument  approach. 

•  Chapter  IV  presents  the  results  of  the  instructor  pilot  (IP)  and  RP  surveys 
and  describes  the  process  of  developing  a  prototype  trainer. 

•  Chapter  V  describes  the  informal  feedback  from  fellow  aviators  and 
includes  a  summary  of  conclusions  and  future  works  related  to  this  study. 
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II.  BACKGROUND 


A.  INTRODUCTION 

The  terminal  environment  of  aviation  is  very  dynamic  and  requires  an  aviator  to 
maintain  an  elevated  level  of  SA.  It  could  also  be  argued  that  aviators  must  maintain  a 
higher  level  of  SA  than  air  traffic  control  and  ground  personnel  in  this  environment.  Even 
under  the  most  ideal  conditions,  this  environment  can  become  overwhelming  for  even  the 
most  experienced  aviators.  A  less  than  appropriate  level  of  SA  could  lead  to  disastrous 
results  for  many  of  the  individuals  involved.  SA  is  comprised  of  many  factors,  and  one 
factor  is  often  dependent  upon  another  factor  or  factors. 

B.  SITUATION  AWARENESS 

Situation  Awareness  has  been  widely  studied  by  researcher  Mica  Ends  ley; 
Endsley  (1995)  describes  SA  as  having  three  separate  levels.  Her  definition  of  SA  is  “the 
perception  of  the  elements  in  the  environment  within  a  volume  of  time  and  space,  the 
comprehension  of  their  meaning,  and  the  projection  of  their  status  in  the  near  future”  (p. 
36). 

The  first  level  of  SA  is  the  ability  to  perceive  the  pieces  of  information,  objects, 
etc.  Perception  is  related  to  an  individual’s  knowledge  and  can  be  influenced  by  the 
design  of  the  system  that  supports  SA  (Endsley,  1995,  pp.  36-37).  She  notes  that  if  an 
individual  has  the  necessary  knowledge  and  the  system  is  designed  to  take  advantage  of 
human  information  processing,  the  likelihood  that  the  salient  cues  will  be  perceived  is 
high  (pp.  36-37).  Initially,  an  individual’s  knowledge  needs  to  be  at  a  level  that  is 
suitable  to  fly  an  IA  within  the  terminal  environment.  An  aviator’s  knowledge  does  not 
have  to  be  at  an  expert  level,  but  he  or  she  needs  to  know  all  the  vital  information  needed 
to  safely  and  successfully  perform  an  IA.  This  includes  the  declarative  and  procedural 
knowledge  concerning  where  and  how  to  read  an  approach,  frequencies,  navigational  aids 
(NAVAID),  approach  course,  altitude,  course  adjustments,  etc.  Additionally,  an  aviator 
needs  to  know  the  Federal  Aviation  Administration’s  guidelines  to  flying  an  IA  such  as 
clearance  procedures  and  missed  approach  procedures.  If  a  system  is  poorly  designed,  it 
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is  likely  that  an  aviator’s  knowledge  alone  will  not  be  sufficient  enough  to  achieve  or 
maintain  a  high  level  of  SA.  Hence,  a  well-designed  system  that  enables  the  perception  of 
environmental  cues  combined  with  the  requisite  knowledge  will  support  an  individual  in 
successfully  achieving  level  1  SA  and  have  a  foundation  for  the  higher  levels  of  SA. 

Level  2  SA  is  the  comprehension  of  the  elements  within  the  current  situation. 
Comprehension  signifies  one’s  ability  to  take  the  perceived  information  from  level  1  and 
piece  that  information  together  in  order  to  create  a  complete  mental  picture  of  the  world 
in  context  of  the  operating  environment  (Endsley,  1995,  p.  37).  Simply  put,  level  2  SA  is 
the  ability  to  understand  the  current  state  of  the  perceived  elements  in  the  environment. 
Being  able  to  create  this  mental  picture  is  a  crucial  aspect  of  developing  and  maintaining 
a  high  level  of  SA  within  any  given  situation. 

What  is  considered  the  highest  level  of  SA,  level  3  SA,  is  one’s  ability  to 
accurately  simulate  the  future  activities  of  the  elements  perceived  within  the  context  of 
the  individual’s  level  2  SA  of  the  world  (Endsley,  1995,  p.  37).  Beyond  perceiving  and 
comprehending  relevant  information,  Level  3  SA  is  the  convergence  of  “comprehending 
the  meaning  of  that  information  in  an  integrated  form,  comparing  it  with  operator  goals, 
and  providing  projected  future  states  of  the  environment  that  are  valuable  for  decision 
making”  (p.  37). 

Endsley  and  Garland  (2000)  state  that  the  proposed  levels  of  SA  describe  a  model 

for  acquiring  information,  processing  it,  and  applying  it  accurately.  They  claimed  an 

individual’s  goals  are  what  determine  what  would  be  considered  appropriate  SA.  The  two 

reported  that  if  goals  change,  what  comprises  appropriate  SA  might  need  to  be  adjusted 

just  as  well.  If  maximal  performance  is  desired,  an  individual  must  strive  to  have  the 

highest  possible  SA  in  order  to  support  the  optimal  decisions  making  on  the  associated 

tasks,  their  study  showed.  However,  desired  outcome  expectations  can  have  an  impact  on 

these  goals.  An  individual  may  want  to  perceive  information  in  a  different  manner  than  is 

required  because  they  were  expecting  something  different  according  to  Endsley  and 

Garland.  According  to  the  authors,  it  is  preferable  to  have  appropriate  expectations  so 

that  when  anticipated  information  is  perceived  it  can  be  processed  effectively.  However, 

it  is  more  important  to  have  the  ability  to  recognize  that  the  information  is  not  as 
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expected  and  the  ability  to  process  it  properly  per  their  study.  Complexity  of  a  situation 
can  be  related  to  the  goal.  In  Endsley’s  1995  work,  she  states  that  if  the  goal  is  too 
difficult,  the  complexity  of  a  situation  could  become  overwhelming  and  detract  from  an 
individual’s  ability  to  maintain  sound  SA  (Endsley,  1995,  p.  53).  This  is  especially 
relevant  for  situations  in  which  an  overall  goal  is  comprised  of  numerous  sub  goals. 

SA  is  the  main  precursor  to  decision-making.  Yet,  many  factors  occur  between 
SA  and  good  decision-making  (Endsley  &  Garland,  2000,  pp.  4-5).  An  individual  may 
have  a  high  SA,  but  their  performance  may  be  poor  due  to  circumstances  outside  of  their 
control  such  as  a  faulty  user  interface  or  lack  of  information  from  third  parties.  In 
addition,  the  information  available  to  an  individual  as  they  developed  their  three  levels  of 
SA  may  be  insufficient,  and  the  individual’s  perception  of  the  situation  can  become 
significantly  subpar.  The  key  takeaway  is  that  “SA,  decision-making  performance  can  be 
seen  for  theoretical  purposes  to  be  distinct  stages  that  can  each  affect  the  other  in  a 
circular  ongoing  cycle,  yet  which  can  be  decoupled  through  various  other  factors”  (p.  5). 

The  measurement  of  SA  is  meaningful  because  it  “provides  a  useful  index  for 
evaluating  system  design  and  training  techniques  and  for  better  understanding  human 
cognition”  (Endsley  &  Garland,  2000,  p.  17).  This  notion  is  essential  because  if  the  goal 
is  to  achieve  a  high  level  of  SA  than  it  is  crucial  that  the  system  design  provides  this 
opportunity.  There  is  not  a  level  of  SA  that  is  required  for  each  situation  but  having  an 
appropriate  level  of  SA  is  valuable.  The  key  though  is  that  the  information  that  is 
available  needs  to  be  assessed  properly  so  that  a  solid  foundation  of  SA  can  be 
developed.  Additionally,  maintaining  SA  is  a  dynamic  function  that  is  constantly 
changing,  and  this  requires  an  individual  to  dynamically  process  the  ever-changing 
information  and  process  it  to  the  best  of  their  ability  at  that  given  time. 

In  addition  to  building  SA,  an  operator’s  attention  is  another  key  topic  within  the 
SA  realm.  If  an  individual  is  being  attentive,  the  SA  of  that  individual  will  typically  be 
higher  (Endsley  &  Garland,  2000,  p.  9).  However,  it  is  important  to  note  that  even  if  an 
individual  is  being  attentive,  they  may  be  attending  to  the  wrong  cues  and  their  SA  could 
be  faulty  and  ultimately  result  in  the  committing  of  errors. 
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According  to  Endsley  and  Garland  (2000),  working  memory  is  an  activated  subset 
of  long-term  memory.  They  suggest,  “information  from  the  environment  may  be 
processed  and  stored  in  terms  of  the  activated  mental  model  or  schema”  (p.  10).  The 
concept  of  a  mental  model  is  something  that  an  individual  develops  in  order  to  visualize 
the  information  and  process  it.  This  concept  is  believed  to  be  how  an  individual  builds 
SA.  These  mental  models  are  used  within  long-term  memory  and  working  memory.  It  is 
this  mental  model  that  “is  the  underlying  knowledge  that  is  the  basis  of  SA”  (p.  12). 
Again,  according  to  Endsley  and  Garland,  the  level  of  SA  depends  on  how  good  a  mental 
model  an  individual  develops.  Applying  Endsley  and  Garland’s  three  levels  of  SA  can 
develop  this  mental  model.  If  executed  poorly,  an  individual  can  develop  a  poor  mental 
model  and  the  results  of  the  situation  could  be  drastic.  Parasuranam,  Sheridan,  and 
Wickens  (2008)  state  that  SA  is  not  a  choice  nor  is  it  general  knowledge  of  material. 
However,  SA  can  be  measured  by  the  performance  of  an  individual  conducting  a  task  or 
multiple  tasks  by  assessing  their  choices,  and  it  is  these  studies  that  have  proven  that  SA 
is  a  quantifiable  instrument  in  the  construct  of  cognitive  engineering  according  to  the 
authors. 

Stress  and  workload  can  be  related  when  analyzing  SA.  Every  person  has  a 
different  capacity  for  stress,  but  a  level  of  stress  is  necessary  in  order  to  perform  at  an 
acceptable  level  (Endsley,  1995,  p.  52).  In  addition,  individuals  have  different  capacities 
for  workload.  If  the  workload  becomes  too  much  for  that  individual,  the  stress  will 
increase,  and  an  individual’s  SA  will  begin  to  decrease  (pp.  52-53).  This  type  of  human 
factor  can  impact  the  SA  of  an  individual,  and  it  is  that  individual’s  capability  that 
impacts  how  stress  affects  the  situation. 

An  important  aspect  of  Endsley’s  work  is  the  concept  of  automaticity,  which  is 
defined  as  the  automatic  processing  of  information  with  minimal  effort.  In  essence,  an 
individual  completes  required  procedures  without  using  too  much  attention  (Endsley, 
1995,  p.  45).  This  is  a  significant  concept  because  it  explains  the  goal  of  this  prototype. 
For  a  USMC  KC-130J  aviator  executing  an  IA,  the  basic  procedures  will  never  change, 
yet  every  IA  is  different. 
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A  key  element  of  the  SA  of  a  USMC  KC-130J  aviator  is  the  use  of  the  aircraft’s 
automation  within  the  cockpit.  Over  the  past  few  decades,  automation  has  increased 
dramatically.  Parasuraman  et  al.  stated  that  as  automation  has  increased,  the  trust  in  the 
automation  has  increased,  and  this  trust  stems  from  the  user’s  knowledge  of  the 
automation  and  proper  manipulation  of  the  automation.  However,  complacency  can  cause 
an  individual  to  have  too  much  trusted  in  the  automation,  which  can  have  negative 
results.  In  addition,  an  individual  with  not  as  much  knowledge  as  necessary  can  rely  on 
the  automation  too  much  and  have  negative  results  as  well  according  to  Parasuraman  et 
al.  It  is  vital  that  aviators  use  the  automation  efficiently  because  “working  with 
automation  have  been  found  to  have  a  diminished  ability  to  detect  system  errors  and 
subsequently  perform  tasks  manually  in  the  face  of  automation  failures  as  compared  with 
manual  performance  on  the  same  tasks”  (Endsley,  1995,  p.  53).  Conversely,  some 
research  has  found  that  automation  can  benefit  SA  because  “automation  that  reduces 
unnecessary  manual  work  and  data  integration  required  to  achieve  SA  may  provide 
benefits  to  both  workload  and  SA”  (p.  54).  Trust  in  automation  is  widely  dependent  upon 
the  individual  and  what  their  knowledge  is  of  the  necessary  material  and  proper  execution 
of  the  automation  (Parasuraman  et  al.,  2008,  p.  151). 

C.  KNOWLEDGE 

Within  the  aviation  domain,  there  are  key  types  of  learning  that  occur:  procedural 
and  conceptual  (Dattel,  Durso,  &  Bedard  2009,  p.  1964).  Procedural  “emphasizes  the 
step-by-step  actions  and  sequences  required  to  complete  a  task”  while  “conceptual 
emphasizes  the  interrelationship  of  the  system  components”  (pp.  1964-1965).  In 
addition,  Cooke,  Salas,  Cannon-Bowers,  and  Stout  (2000)  stated  that  declarative 
knowledge,  which  consists  of  facts,  figures,  rules,  relations  and  concepts  in  a  task 
domain,  is  another  knowledge  (p.  157).  Before,  conceptual  and  procedural  knowledge  is 
explained,  declarative  knowledge  has  an  overarching  impact  on  the  other  two  types  of 
knowledge  because  it  is  imperative  that  an  aviator  have  that  type  of  knowledge 
memorized  before  evening  taking-off.  In  order  to  decipher  between  how  conceptual  and 
procedural  knowledge  work  within  aviation,  the  two  techniques  can  be  applied  to 

different  aspects  of  the  aviation  domain.  For  example,  procedural  is  beneficial  when  a 
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typical  procedure  needs  to  be  followed  in  order  to  successfully  accomplish  a  task  such  as 
normal  checklists,  communication  calls,  and  landings.  If  at  any  time  during  the  situation 
it  becomes  atypical,  conceptual  knowledge  becomes  more  beneficial  than  procedural 
because  a  “conceptual  knowledge  of  the  interrelationships  of  the  system  components  may 
successfully”  remedy  the  atypical  situation  (Dattel  et  al.,  2009,  p.  1967). 

Dattel  et  al.  (2009)  conducted  an  experiment  that  involved  separate  groups  of 
novice  aviators  learning  aviation  knowledge  such  as  course  rules  and  landing  pattern 
traffic  via  conceptual  and  procedural  knowledge.  At  the  conclusion  of  the  experiment, 
Dattel  et  al.  determined  that  the  conceptual  knowledge  technique  is  better  for  aviation 
skills.  Particularly,  Dattel  et  al.  concluded  that  for  non-complex  “unfamiliar  and  atypical 
conditions”  conceptual  learning  is  better  than  procedural  (p.  1967). 

D.  PART-TASK  TRAINING 

Part-task  training  is  an  emerging  phenomenon  that  is  becoming  a  crucial 
supplemental  resource  in  many  training  fields  (Stanney,  Mourant,  and  Kennedy  1998,  p. 
327).  PTTs  have  offered  a  more  convenient  and  way  of  accessing  information  and 
communication  (John,  Phillips,  Cenydd,  Coope,  Carleton-Bland,  Kamaly-Asl,  and  Gary, 
2015,  p.  156).  For  example,  neurosurgeons  in  the  United  Kingdom  have  developed  an 
application  for  the  iPad  in  order  to  train  neurosurgeon  trainees  with  vital  procedures  of  a 
particular  brain  surgery  in  order  to  reduce  a  patient’s  possible  exposure  to  additional  risk 
(p.  155).  Initial  results  have  demonstrated  promise  to  the  positive  effect  PTT  can  have  in 
training  neurosurgeon  trainees  with  brain  surgery  procedures.  These  finding  suggest  that 
using  a  PTT  as  a  supplemental  resource  can  have  a  significant  impact  on  future  training 
within  many  fields. 

Part-task  training  can  provide  the  user  with  additional  confidence,  situation 
awareness,  and  knowledge  to  successfully  execute  important  steps  within  a  crucial 
procedure.  The  Marine  Corps  has  recognized  the  idea  of  using  modeling  and  simulation 
in  order  to  maintain  the  skills  of  Marines  in  regards  to  their  duties  (Neller,  USMC,  2016). 
In  particular,  the  Training  and  Education  Command  of  the  Marine  Corps  has  conducted 
research  into  Deployable  Virtual  Training  Environment  (DVTE)  for  the  purpose  of 
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sustaining  Marines’  readiness  (Brannon  &  Villandre,  2002,  p.  15).  It  is  suggested  here 
that  the  deliberate  creation  and  use  of  a  PTT  for  training  will  continue  to  meet  the  needs 
of  the  Marine  Corps  and  could  begin  to  turn  the  tide  of  their  deployment  and 
underutilization  for  training. 

There  are  some  important  human  factor  elements  that  need  to  be  examined  when 
developing  a  PTT.  Stanney,  Mourant,  and  Kennedy  (1998)  state  that  it  is  important  to 
consider  how  to  assess  a  VE  in  order  to  determine  the  impact  on  training  that  has 
occurred  through  the  use  of  the  device.  The  desire  is  to  have  the  outcome  of  the  training 
with  the  VE  device  be  no  less  than  the  outcome  of  its  predecessor’s  training  according  to 
the  authors.  In  other  words,  “the  use  of  a  VE  should  improve  task  performance  when 
transferred  to  the  real-world”  (p.  330).  Furthermore,  there  is  also  the  appeal  to  have  the 
VE  device  easy  to  navigate  and  use  (p.  329).  In  order  to  accomplish  these  goals,  it  is 
recommended  that  the  user  experience  appropriate  levels  of  immersion  and  presence 
when  using  a  VE  system  according  to  Slate  and  Wilbur  (1997).  Presence  is  defined  as  a 
“psychological  sense  of  being  somewhere  else  than  in  one’s  immediate  physical  world” 
while  “immersion  is  a  degree  to  which  the  technical  elements  of  VE  solutions  envelop  the 
users”  (Slater  and  Wilbur,  1997,  pp.  606-607). 

Stanney  et  al.  (1998)  suggests  that  a  positive  transfer  of  training  can  occur  as  long 
as  original  learning  occurs.  In  VE,  original  learning  can  occur  if  the  immersion  and 
presence  are  representative  of  the  task  environment  (p.  332).  Key  aspects  of  human 
performance  that  should  be  considered  in  the  design  of  the  VE  device  are  the  “abilities 
and  limitations  of  human  sensory  and  motor  physiology”  (p.  334).  When  referring  to  the 
learning  aspect  of  VEs,  “there  are  strong  scientific  reasons  why  the  active  engagement 
available  in  VE  games  rather  than  passive  action  of  television  watching  can  have  better 
retention  of  learned  skills”  (p.  344). 

E.  SUMMARY 

This  chapter  described  SA,  aspects  of  SA  that  can  impact  situations,  and  how 
factors  such  as  presence,  immersion,  usability,  and  performance  assessment  impact  the 
acceptability  of  the  use  of  VEs  for  military  training.  Additionally,  the  types  of  knowledge 
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(procedural,  declarative,  and  conceptual)  were  addressed  as  well  as  their  relation  to 
aviation.  Finally,  a  case  is  made  on  how  PTT  could  be  used  as  a  viable  tool  for  enhancing 
SA,  and  the  type  of  learning  that  is  required  having  a  positive  transfer  of  skills  and 
knowledge  into  the  intended  environment. 
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III.  TASK  ANALYSIS 


A.  INTRODUCTION 

An  aviator  should  have  a  standard  practice  for  setting-up  and  performing  the 
necessary  procedures  of  an  IA.  Regardless  of  what  special  situation  or  even  emergency 
arises  during  any  particular  IA,  the  aviator  with  a  standard  practice  for  an  IA  has  the 
potential  to  successfully  handle  these  situations  and  fly  the  approach  successfully.  It  is 
the  repetition  of  a  standard  practice  that  has  a  power  to  permit  aviators  the  ability  to 
maintain  a  high  level  of  SA  throughout  the  approach  according  to  Fowlkes,  Dwyer,  Oser, 
and  Salas  (1998). 

The  need  to  understand  the  tasks  that  are  to  be  simulated  requires  an  analysis  of 
the  job.  Naval  Education  and  Training  Command’s  (NETC)  Job  Duty  Task  Analysis 
(JDTA)  Manual  (NETC,  2011)  provides  a  framework  in  which  the  task  analysis  (TA) 
divides  a  task  into  separate  steps  and  identifies  important  characteristics  of  each  step.  The 
initial  step  is  to  identify  the  components  and  sequence  of  steps  of  the  task.  Next,  the 
conditions  under  which  the  task  will  be  performed  must  be  identified  along  with  any 
triggers  or  cues  that  will  keep  the  sequence  progressing.  Finally,  a  performance  level 
must  be  administered  with  each  task  so  that  the  individual  executing  the  task  can  be 
evaluated. 

The  TA  for  KC-130J  IAPs  was  conducted  in  context  of  SA,  knowledge,  and  PTT 
described.  The  following  section  describes  the  approach  and  outcomes  of  the  TA  that  was 
used  to  create  the  prototype  system. 

B.  SITUATION  AWARENESS,  KNOWLEDGE,  AND  INSTRUMENT 

APPROACH  PROCEDURES 

The  KC-130J  community  within  the  USMC  has  a  common  practice  for  their 
aviators.  First,  aviators  are  to  request  and  receive  the  weather  report  at  the  necessary 
airfield  (Department  of  the  Navy,  2014).  The  weather  report  is  their  initial  SA  builder  and 
should  be  retained  throughout  the  approach.  The  next  step  is  for  the  aviator  to  select  the 
correct  IA  to  fly,  and  an  aviator’s  conceptual  knowledge  should  assist  in  choosing  the 
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best  IA.  The  next  several  steps  of  conducting  an  IA  have  many  similar  aspects  that 
require  the  aviator’s  procedural  knowledge  to  be  sound.  For  example,  an  aviator  needs  to 
set-up  the  cockpit  instruments  properly,  have  the  correct  decelerating  airspeed, 
descending  altitude,  and  the  correct  heading.  These  are  just  a  few  of  the  steps  to  flying  an 
IA,  and  they  often  occur  at  the  same  time.  Rogers,  Maurer,  Salas,  and  Fisk,  (1997)  state 
that  it  is  theorized  that  tasks,  or  parts  of  tasks,  can  be  trained  to  become  habitual  or 
automatized.  When  achieved,  automaticity  results  in  the  performance  of  those  tasks  or 
subtasks  with  little  cognitive  effort.  An  aviator’s  automaticity  can  be  developed  if  the 
same  practice  is  used  when  conducting  an  IA  and  can  become  paramount  in  the  ability  of 
the  aviator  to  maintain  a  sound  SA. 

Throughout  the  entire  IA,  an  aviator  will  rely  on  what  is  referred  to  as  the  three 
separate  levels  of  SA,  (1)  perception  of  the  elements  within  the  environment;  (2) 
comprehension  of  the  current  situation;  and  (3)  projection  of  future  status,  that  Endsley 
discussed  (Endsley,  1995,  pp.  36-37).  While  the  procedures  of  an  IA  should  be  constant 
for  every  approach,  other  elements  such  as  crew  relationships,  weather,  and  airfield 
features  are  different,  which  project  different  future  status.  Successful  landings,  despite 
the  potential  problems  presented  by  these  variables,  are  built  on  a  strong  foundation  of 
conceptual  knowledge.  These  three  levels  tie  into  many  sublevels  such  as  time,  space, 
and  decision-making,  and  those  are  vital  aspects  of  every  IA. 

The  SA  of  an  aviator  while  flying  an  approach  can  be  impacted  by  many  factors 
such  as  attention,  perception,  working  memory,  and  long-term  memory.  Attention  is 
needed  in  order  to  properly  perceive  the  information  and  apply  the  appropriate  memory 
(Endsley,  1995,  pp.  40-41).  As  it  is  further  clarified,  working  memory  conducts  the 
active  processing  of  information  whereas  long-term  memory  uses  stored  information  to 
counter  any  limitations  of  working  memory. 

For  KC-130J  aviators,  the  impact  of  automation  plays  an  important  role  in  their 
development  and  maintenance  of  SA.  The  aircraft  is  capable  of  complete  autopilot,  which 
consists  of  navigation,  speed  control,  and  altitude  control.  Endsley  points  out  three 
cmcial  reasons  as  to  how  operators  of  automated  systems  may  have  slower  reactions 
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times  to  problems  and  longer  recovery  times  (Endsley,  1995,  p.  54).  The  three  reasons 
are: 

a  loss  of  vigilance  and  increase  in  complacency  associated  with  the 
assumption  of  a  monitoring  role  under  automation,  the  difference  between 
being  an  active  processor  of  information  in  manual  processing  and  a 
passive  recipient  of  information  under  automation,  and  a  loss  of  or  change 
in  the  type  of  feedback  provided  to  operators  concerning  the  state  of  the 
system  under  automation  (p.  54). 

Considering  these  circumstances,  one  can  argue  that  aviators  utilize  more 
cognitive  resources  to  maintain  the  required  level  of  SA  during  automation  than  when 
manually  flying  the  aircraft  because  being  actively  engaged  in  the  task  can  assist  with 
keeping  the  attention  and  perception  high. 

C.  INSTRUMENT  APPROACH  PROCEDURES 

This  section  will  focus  on  the  prototype  questions  and  cues  used  as  the  basis  for 
the  prototype  that  was  developed  for  this  thesis.  These  questions  and  cues  were  derived 
from  the  high  level  TA  for  an  IAP.  Additionally,  these  are  color  coded  to  correspond  to 
the  TA  (see  Appendix  B).  For  brevity,  this  section  will  focus  on  PF  information  since  the 
PM  information  is  nearly  identical. 

1.  Prototype  Question  TA 

Prototype  question  TA  is  from  the  Department  of  the  Navy’s  (2014)  Naval  Air 
Training  and  Operating  Procedures  Standardization  Flight  Manual  selects  sets  of  items 
that  are  part  of  NATOPS  task  description  have  been  expressed  in  the  prototype  system 
via  questions-and-answers  as  part  of  the  user  interface.  These  questions  require  the  user 
to  recognize  the  correct  sequence  of  steps  as  presented  on  the  corresponding  instrument 
panels.  Segments  of  NATOPS  task  description  that  are  presented  as  question-and  answer 
form  of  interface  include: 

Step  1.  Review  Pressurization  Panel. 

Step  2.  Review  Anti-Ice/De-ice  Panel. 

Step  3.  Review  HUD. 
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Step  4.  Review  CNBP  Menu. 

Step  5.  Review  AMU  Menu. 

Step  6.  Review  REF/MODE  panel. 

Step  7.  Review  CNI-MU. 

Step  8.  Fasten  Belts/Harnesses. 

Step  9.  Complete  Approach  Brief  at  appropriate  time. 

2.  Prototype  Visual  Cues  TA 

Prototype  visual  cues  TA  is  from  the  Department  of  the  Navy’s  (2014)  Naval  Air 
Training  and  Operating  Procedures  Standardization  Flight  Manual.  There  are  several 
visual  cues  that  an  aviator  must  recognize  and  act  accordingly  when  flying  an  IA.  The 
prototype  presents  several  of  those  cues  by  mirroring  the  aircraft’s  layout  of  the  heads  up 
display  (HUD)  on  the  user  interface.  Visual  cues  address  the  following  segments  of 
NATOPS  task  description.  These  include: 

Select  the  appropriate  NAVAID  when  cleared  by  ATC. 

Maintain  awareness  of  the  DME. 

Maintain  awareness  of  the  altitude. 

Maintain  awareness  of  heading. 

3.  Prototype  Audio  Cues  TA 

Prototype  audio  cues  TA  is  from  the  Department  of  the  Navy’s  (2014)  Naval  Air 
Training  and  Operating  Procedures  Standardization  Flight  Manual.  Throughout  an  IA, 
radio  calls  are  constant  and  provide  valuable  cues  that  an  aviator  needs  to  recognize.  The 
scenario  that  has  been  presented  in  our  prototype  system  has  a  number  of  common  radio 
calls  that  an  aviator  should  listen  to  and  know  what  action  is  expected  to  happen  and 
when  to  do  it.  The  following  list  presents  elements  of  NATOPS  task  description  that  were 
integrated  as  audio  cues  within  the  prototype. 

Listen  to  the  weather  report  to  determine  which  IA  to  fly. 
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Execute  an  approach  brief  once  the  PM  has  completed  flows. 
Listen  to  ATC’s  clearance  for  the  approach. 

Listen  for  the  mandatory  call  outs  during  the  approach. 


D.  SUMMARY 

This  chapter  discussed  the  outcomes  of  the  TA,  which  was  conducted  in  context 
of  SA  and  requisite  knowledge  for  KC-130J  IAs.  This  TA  serves  as  the  basis  of  the 
prototype  and  the  scenario  created  to  represent  an  IA  for  this  thesis. 
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IV.  DESIGN  AND  DEVELOPMENT  OF  THE  PROTOTYPE 

TRAINING  SYSTEM 


A.  INTRODUCTION 

Before  a  design  and  development  of  a  prototype  system  was  approached,  a  survey 
of  KC-130J  subject  matter  experts  (SMEs)  was  conducted.  The  objective  of  this  work 
was  to  capture  information  about  possible  gaps  in  training,  the  issues  and  trends  they 
identified  with  performance  of  the  aviators  during  FAM  stage  (with  special  emphasis  on 
IAP),  and  their  suggestions  on  how  a  new  PTT  could  fit  in  and  benefit  the  current 
training  regimen.  It  is  important  to  obtain  the  current  pulse  of  the  fleet  while  developing 
such  a  device  because  it  provides  a  review  of  the  most  up-to-date  insights  and  necessities 
as  they  are  seen  by  the  warfighters  i.e.,  the  actual  users  of  current  and  future  systems. 

B.  SURVEY  OF  A  USER  BASE 

The  use  of  human  subjects  in  surveys  that  includes  data  collection  from  those 
individuals  and  analysis  of  the  same  data  sets,  dictates  applying  institutional  review  board 
(IRB)  protocol  and  its  adopted  requirements  and  procedure.  In  the  case  of  this  thesis,  it 
required  permission  to  proceed  with  the  study  from  an  IRB  Committee  at  NPS  and  within 
the  U.S.  Marine  Corps;  the  latter  required  approval  from  the  survey  approval  office  and 
the  human  research  protection  office  of  the  U.S.  Marine  Corps.  (See  Appendix  C) 

1.  Subject  Matter  Expert  Input 

Two  KC-130J  SME  were  consulted  regarding  different  issues  in  this  domain  - 
their  insights  and  feedback  helped  us  develop  the  entire  survey  that  would  be  delivered  to 
the  target  user  base.  The  questions  for  the  SMEs  were  rather  generic  and  open-ended 
about  training  IAPs  for  the  KC-130J;  their  responses  allowed  us  to  generate  more  precise 
questions  for  the  official  online  survey.  The  full  list  of  questions  submitted  to  the  SMEs 
is  in  Appendix  A;  only  the  most  significant  responses  given  by  the  SMEs  are  discussed  in 
this  section. 
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The  SMEs  reported  that  they  observed  RPs  having  problems  with  basic  IAP.  One 
SME  commented  that  RPs  rely  too  much  on  the  automation  of  the  KC-130J,  which  could 
lead  to  problems  escalating  before  the  RP  even  recognizes  the  existence  of  a  problem.  A 
common  theme  amongst  the  SME’s  responses  was  that  repetitive  exposure  to  the  IAP  via 
practice  such  as  chair  flying  could  be  very  beneficial.  Another  valuable  point  commented 
by  a  SME  was  that  RPs  struggle  with  complex  approaches  or  busy  airports.  These 
statements  were  not  acquired  to  validate  the  purpose  of  this  thesis  but  rather  to  confirm 
potential  value  of  repetitive  chair  flying.  The  SMEs’  replies  confirmed  that  a  PTT  could 
be  used  as  a  supplemental  training  tool,  as  it  was  outlined  in  this  thesis.  That  type  of 
training  solutions  can  be  seen  as  essential  tools  for  enabling  additional  training 
opportunities  that  are  needed  to  maintain  SA  at  an  appropriate  level. 

2.  Online  Survey:  Procedures  and  Results 

The  LimeSurvey  system  was  used  as  an  instrument  that  helped  generate  online 
surveys  for  RPs  and  IPs.  This  system  allowed  a  link  to  a  web-based  application  be  sent  to 
potential  survey  takers,  thus  eliminating  the  need  for  subjects  to  come  to  a  laboratory  and 
fill  the  physical  questionnaire. 

Participation  in  all  surveys  was  voluntary  basis.  Unfortunately,  only  two  IPs 
responded  to  our  call,  nevertheless  valuable  information  was  acquired  even  with  a  small 
number  of  participants.  Questions  for  the  RPs  addressed  several  topics  that  included 
current  studying  (training)  practices,  subjects’  perceptions  about  current  IA  practices, 
trends  in  students’  performance,  and  desired  capabilities  that  future  PTT  could  provide.  A 
full  list  of  survey  questions  is  in  Appendix  A  and  only  the  most  significant  replies  of  the 
IP  survey  are  summarized  in  this  section. 

The  two  IP  survey  results  collected  in  this  study  reported  a  common  issue 
identified  in  procedural  execution.  Procedural  execution  is  not  a  lack  of  procedural 
knowledge;  in  this  content  it  refers  to  a  lack  of  IAP  experience  such  as  performing  the 
flows  efficiently  and  accurately.  RPs  tend  to  take  too  much  time  performing  these  flows, 
which  results  in  a  RP  getting  well  behind  the  approach’s  timeline.  Additionally,  the  IP 
survey  reveals  that  RPs  tend  to  struggle  with  communications  and  maintaining  SA.  Both 
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responses  address  the  importance  of  chair  flying.  The  survey  also  addressed  questions 
pertaining  to  features  desired  in  a  future  PTT;  that  discussion  will  be  addressed  in 
Chapter  V.  As  stated,  the  RPs’  struggles  with  flow  execution  were  used  as  guidance 
during  the  design  phase  of  the  prototype.  For  example,  the  in-range  flows  are  one  of  the 
few  items  that  are  visually  depicted  within  the  prototype  so  that  a  RP  can  visualize  their 
proper  sequence. 

C.  CONCEPTUAL  FRAMEWORK 

The  overall  conceptual  framework  for  this  PTT  is  very  simple.  The  questions  are 
called  from  a  JavaScript  Object  Notation  file,  which  allows  the  user  to  answer  the 
questions  presented  in  the  scenario.  At  the  end  of  the  scenario  execution,  the  IP  can  view 
the  results  of  how  the  user  answered  the  questions  from  a  file  once  the  scenario  is 
complete.  Figure  1  shows  elements  of  the  conceptual  framework  that  has  been  adopted 
for  this  PTT. 


Answer 

Questions 


Figure  1.  Conceptual  framework. 


D.  PROGRAMMING  AND  DEVELOPMENT  ENVIRONMENT 

Since  this  is  a  prototype  training  system  at  a  very  basic  level,  many  aspects  of  the 
coding  were  hard  coded,  which  does  not  permit  any  of  the  scenario  to  be  altered.  The 
reasoning  behind  this  concept  is  based  on  two  primary  reasons:  1)  it  is  a  proof-of-concept 
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2)  the  prototype  addresses  the  basic  IAP  and  radio  calls  a  RP  should  know  and  expect  to 
hear. 


1.  Unity  Game  Engine  and  C# 

The  Unity  game  engine  was  used  to  develop  the  scenario  since  it  has  vast 
capabilities  with  the  most  important  being  that  it  is  compatible  with  multiple  operating 
systems.  Unity  was  primarily  developed  to  assist  with  the  development  of  video  games 
and  simulations.  In  addition,  Unity  is  free  to  download  from  the  Internet,  which  makes  it 
easily  accessible.  These  characteristics  make  Unity  a  viable  choice  when  selecting  a 
program  to  use  for  an  effort  such  as  this  thesis.  Figure  2  shows  a  segment  of  the  terrain 
used  in  the  PTT  as  seen  inside  the  Unity  interface. 


Figure  2.  Unity  game  engine. 


2.  Images  of  the  PTT  Interface 

Figures  3-7  depict  different  elements  of  the  PTT  interface  that  the  user  will 
experience  during  the  scenario.  The  images  display  different  aspects  of  the  scenario 
ranging  from  the  instrument  panels  with  their  questions  to  how  the  terrain  looks  at 
different  altitudes. 
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Figure  3.  Screen  shot  at  6000  feet. 


Which  elements  do 
you  check? 


Figure  4.  Screen  shot  of  instrument  panel  and  question. 
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At  what  DME  must 
you  be  have  all 


Figure  5.  Screen  shot  of  approach  plate  and  question. 


Figure  6.  Screen  shot  at  3677  feet. 
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Figure  7.  Screen  shot  of  airfield  at  Twenty-nine  Palms 


3.  Auditory  Cues 

The  Audacity  application  was  used  to  record  radio  calls,  which  commonly  occur 
during  an  IA.  In  order  to  increase  realism  within  the  environment,  engine  noise  was 
added  to  the  scenario.  This  audio  sample  was  downloaded  from  a  free  sound  library 
(freesound.com).  The  use  of  the  Unity  development  environment  allowed  for  the 
relatively  easy  addition  of  the  required  auditory  cures;  simple  drag  and  drop  action  is  all 
that  is  required  to  add  this  capability  to  the  final  user  interface. 

E.  DEVELOPMENT 

The  initial  idea  was  that  this  prototype  training  system  would  replicate  a  very 
realistic  ability  to  practice  IAP,  but  through  the  tasks  analysis  and  SME  input,  it  became 
evident  that  a  system  of  this  complexity  was  not  needed  since  the  goal  of  this  prototype 
was  to  create  a  supplemental  tool  to  serve  as  a  PTT.  Hence,  the  scope  of  the  prototype 
was  narrowed  down  to  address  the  procedures  and  situation  awareness  associated  with 
IA.  The  user  could  practice  the  IA  task  as  often  as  it  was  desired,  allowing  those  skills  to 
become  second  nature  so  that  an  appropriate  level  of  SA  could  be  maintained  while 
executing  IAPs. 
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The  prototype  training  system  was  designed  to  support  the  execution  of  a  scenario 
from  the  beginning  of  an  IA  until  just  before  the  touchdown  upon  an  airfield.  A  minimal 
user  input  is  required  except  for  answering  a  set  of  questions  that  test  user’s  SA  and 
knowledge.  The  prototype  depicts  the  first  person  view,  and  it  adds  an  additional  visual 
layer  that  replicates  a  look  and  feel  of  the  HUD  of  the  KC-130J  (Figure  6).  The  scenario 
used  in  the  system  addresses  all  the  steps  described  in  the  NATOPS  documentation;  the 
same  steps  must  occur  for  a  normal  IA  along  with  the  standard  cockpit  and  radio  calls. 

F.  MODES  OF  USE 

The  prototype  can  be  used  in  three  modes  or  configurations:  tablet,  laptop,  or 
large  screen.  All  three  modes  of  use  offer  different  aspects  that  can  enhance  general 
capability  of  the  prototype.  The  tablet’s  advantage  is  that  an  individual  can  practice  IAP 
anywhere  at  any  time.  A  laptop  can  be  used  amongst  a  small  group.  A  large  screen  device 
can  be  used  in  an  instructor-led  classroom.  Figures  8-10  depict  the  different  modes. 


Figure  8.  Tablet  mode. 
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Figure  9.  Laptop  mode. 


Figure  10.  Large  screen  mode. 
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G.  INFORMAL  DEMONSTRATION 

An  informal  demonstration  of  the  prototype  was  conducted  with  the  goal  to 
acquire  valuable  feedback  that  could  be  applied  to  future  work. 

1.  Study  Design 

The  demonstration  population  consisted  of  aviators  currently  attending  the  Naval 
Postgraduate  School  (NPS).  A  request  was  submitted  to  the  registrar’s  office  to  acquire  a 
list  of  Navy  and  Marine  aviators  so  that  their  presence  could  be  requested  for  the 
demonstration.  Of  the  thirty-five  aviators  contacted,  eight  participated  in  the 
demonstration. 

For  the  demonstration,  a  brief  explanation  of  the  thesis  and  prototype  was  given. 
Upon  completion  of  the  brief,  the  scenario  was  shown  to  the  aviators.  The  scenario  lasted 
about  eight-and-a-half  minutes.  At  the  conclusion  of  the  demonstration,  the  participants 
handed  in  written  answers  to  the  questionnaire  (see  Appendix  A),  thanked  for  their 
participation,  and  departed  the  demonstration  area. 

2.  Results 

The  insights  gained  during  the  demonstration  coincide  with  recommendations 
from  the  IP  LimeSurvey: 

•  Ability  to  develop  a  scenario  so  that  a  RP  can  be  evaluated  for  a 
particular  portion  of  an  IA.  Currently,  the  prototype  executes  an 
approach  from  start  to  finish,  which  does  not  allow  the  user  to  focus  on  a 
single  section  of  the  approach. 

•  Provide  feedback  and  a  study  guide.  A  suggestion  collected  in  the  IP 
survey  was  that  the  PTT  could  provide  guidance  for  the  user  and  suggest 
elements  that  needed  to  be  improved  in  order  to  develop  the  required  IAP 
skills. 

•  Ability  to  study  approaches  at  any  airfield.  A  common  comment  from 
the  demonstration  was  a  desire  to  have  the  PTT  execute  and  support  IA  at 
several  airfields  with  multiple  approaches. 

•  Ability  to  select  or  alter  switches/buttons.  Both  the  IP  survey  results  and 
the  demonstration  expressed  a  desire  to  have  the  PTT  provide  the  user  the 
opportunity  to  select  switches  or  buttons  on  the  instrument  panels.  Some 
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of  the  buttons  or  switches  might  require  a  haptic  feedback,  yet  this  would 
only  need  to  be  a  vibration  sensation  similar  to  a  smart  phone’s  vibration 
when  a  button  is  selected.  This  type  of  haptic  feedback  would  permit  the 
device  to  still  be  transportable.  While  including  haptics  could  be 
advantageous  for  the  overall  learning  process,  it  could  also  be  more  than 
what  a  PTT  needs  to  perform.  An  analysis  of  how  passive  haptics  would 
impact  a  PTT  would  need  to  be  examined. 

•  Voice  recognition.  The  radio  calls  are  prerecorded  in  this  prototype,  yet 
many  expressed  a  desire  to  have  a  support  of  voice  recognition  software  so 
that  the  user  could  practice  radio  calls. 

•  Moving  Screen.  The  orientation  of  the  viewpoint,  first  person,  in  the  PTT 
does  not  change,  and  a  suggestion  received  from  SMEs  was  to  introduce 
the  possibility  of  changing  the  orientation  of  the  viewpoint  independently 
so  that  a  section  of  the  cockpit  with  the  instruments  could  be  seen. 

•  Impose  time  constraint  to  scenario.  For  a  more  advanced  PTT,  a 
suggestion  was  to  put  the  user  in  a  time  constraint  scenario  and  emulate 
real  situations. 

•  Pop-up  windows.  During  dead  space  of  the  scenario,  it  has  been 
suggested  to  have  pop-up  questions  or  comments  to  keep  the  user 
engaged. 

H.  SUMMARY 

This  chapter  discussed  different  aspects  of  the  design  and  development  of  the 
prototype  training  system.  This  included  a  survey  of  the  user  base  and  its  results, 
followed  by  a  review  of  conceptual  framework,  the  user  interface  that  was  developed, 
and  finally  a  discussion  of  the  informal  demonstration  and  feedback  gathering  that  was 
done  with  a  help  of  local  SMEs  at  NPS. 
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V.  DISCUSSIONS  AND  CONCLUSIONS 


A.  DISCUSSION 

The  increasing  accessibility  of  low-cost,  off  the  shelf  technologies  presents  an 
opportunity  to  drastically  impact  how  aviation  training  is  accomplished.  Recognizing 
this,  the  thesis  presented  here  looked  to  answer  an  overarching  research  question  and  two 
additional  research  questions  that  support  the  main  question.  First,  would  it  be  feasible  to 
develop  a  VR  system  to  support  training  IAPs  to  improve  situation  awareness?  Next, 
what  if  any,  training  gaps  exist  with  the  training  of  IAPs  and  SA  for  IAs?  Lastly,  and 
based  on  the  former,  what  cues  should  be  included  in  a  PTT  to  support  the  practice  of 
gaining  and  maintaining  SA  during  IAs? 

The  first  question  was  partially  answered.  Through  a  modified  instructional 
systems  design  approach,  this  thesis  demonstrates  that  a  systematic  approach  to  the 
development  of  a  PTT  for  training  IAs  could  be  accomplished  in  a  relatively  short 
amount  of  time  in  a  resource-constrained  environment.  Unfortunately,  due  to 
administrative  delays,  a  training  effectiveness  evaluation  was  not  conducted  and  the 
second  part  of  our  first  research  question  cannot  be  definitively  answered.  However,  post¬ 
demonstration  SME  feedback  suggests  that  the  prototype  developed  for  this  thesis  shows 
promise  for  helping  aviators  in  the  KC-130J  FRD  develop  knowledge  about  IAPs  and 
improve  their  ability  to  rapidly  perceive,  assess,  and  plan  their  actions  during  IAs. 

In  support  of  the  main  question,  the  exploration  of  potential  training  gaps  was  a 
mandatory  step  to  ensure  that  the  effort  of  this  thesis  was  warranted.  Through  this  work, 
we  discovered  that  gaps  do  exist  in  the  form  of  available  opportunities  and  the  fidelity  of 
training  aides.  Further,  it  is  suggested  here  that  a  fundamental  change  in  how  training  is 
considered  may  be  overdue.  While  technology  can  be  considered  merely  the  vessel  in 
which  content  is  delivered,  current  and  future  generations  of  aviators  may  grow  to  expect 
particular  technologies  to  be  part  of  their  experience.  This  is  not  to  minimize  the  value  of 
chair  flying  as  a  form  of  mission  rehearsal  but  to  offer  that  aviators  using  virtual 
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technologies  in  their  everyday  lives  may  have  improved  training  experiences  and 
outcomes  if  these  technologies  are  available  for  their  training. 

Finally,  and  perhaps  most  importantly  for  the  development  of  any  training  system, 
the  identification  and  incorporation  of  the  system  and  environmental  cues  needed  to 
support  training  goals  was  accomplished.  Mistakes  at  this  stage  could  have  disastrous 
results.  Ensuring  elements  such  as  cockpit  displays,  environmental  conditions, 
communications,  time,  and  space  are  accurately  represented  is  critical  for  avoiding 
negative  transfer  of  training  for  aviators.  We  are  confident  that  the  thorough  review  of 
available  publications,  SME  interviews,  and  SME  evaluations  conducted  for  this  thesis 
enabled  us  to  understand  these  cues  and  translate  them  into  the  KC-130J  IA  training 
prototype  created.  In  sum,  this  thesis  was  able  to  achieve  the  goals  as  originally  stated. 
However,  as  with  many  projects  such  as  this,  there  is  still  work  that  must  be 
accomplished  before  a  prototype  could  be  released  to  actually  support  training. 

B.  FUTURE  WORK 

The  general  objective  of  this  thesis  was  very  broad,  and  future  work  in  this 
domain  could  accomplish  that  goal.  An  entire  range  of  PTTs  can  become  valuable 
training  solutions  in  the  future  and  address  a  variety  of  training  gaps.  Scenarios  for  other 
missions  could  be  developed;  this  would  permit  aviators  the  repetitive  opportunities 
needed  to  master  a  plethora  of  skills  required  to  safely  and  successfully  operate  within  the 
KC-130J  community.  In  that  regard,  a  very  similar  concept  of  PTT  could  be  used  for 
other  segments  of  aviation  community.  An  additional  line  of  future  work  could  network 
these  PTTs  together  so  that  separate  communities  could  conduct  joint  mission  rehearsals 
such  as  an  aerial  refueling  mission. 

1.  Knowledge 

This  thesis  focused  on  SA;  however,  there  is  a  significant  amount  of  work  needed 
to  analyze  the  three  types  of  knowledge  that  were  introduced  within  chapter  two.  The 
work  could  focus  on  how  each  type  is  important  to  an  IA  and  the  IAP  of  the  KC-130J  in 
support  of  maintaining  SA.  The  reference  for  this  material  within  this  thesis  is  a  good 
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starting  point  for  knowledge  and  general  aviation,  but  a  focus  of  knowledge  and  the  KC- 
130J  could  be  beneficial  work  for  the  community. 

2.  Eye  Tracking 

A  thesis  that  could  integrate  eye  tracking  system  and  try  to  differentiate  between  a 
novice,  junior  level,  and  SME  executing  an  IA.  The  study  could  be  designed  to  acquire 
understanding  of  what  each  of  those  three  separate  groups  is  looking  at  during  an  IA  and 
provide  valuable  feedback  on  what  to  do  and  what  not  to  do  during  an  IA.  Eye  tracking 
could  also  provide  important  insights  as  to  the  progression  of  a  trainee  through  a 
curriculum.  If  a  trainee’s  scan  pattern  repeatedly  matches  an  expert  model,  it  may  be 
possible  to  advance  them  through  a  training  program  more  rapidly.  Conversely,  it  would 
also  be  possible  to  identify  those  who  may  need  extra  training  opportunities  or  other 
interventions. 

C.  CONCLUSIONS 

The  importance  of  developing  a  PTT  that  can  supplement  current  training  of  IAPs 
cannot  be  understated.  The  value  that  a  PTT  could  be  easily  transported  and  used  at  the 
user’s  leisure  is  significant.  A  PTT  can  fulfill  that  gap  of  repetition  and  allow  an 
individual  to  master  procedural  and  demonstration  knowledge  with  the  ultimate  goal  of 
developing  conceptual  knowledge  and  maintaining  sound  SA. 
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APPENDIX  A.  QUESTIONNAIRE  AND  SURVEYS 


1.  SME  QUESTIONS 

1.  In  your  experience,  what  are  the  types  of  issues  that  pilots  struggle  the  most  with  as 
they  begin  conducting  instrument  approach  procedures  in  the  actual  aircraft?  List 
three  major  issues  that,  in  your  view,  fit  this  description. 

2.  Given  your  experience  with  RPs,  how  well  do  you  think  that  the  majority  of  them 
perform  instrument  approach  procedures? 

1  =  it  is  well  beyond  minimal  standards 

2  =  it  is  beyond  minimal  standards 

3  =  it  is  within  minimal  standard 

4  =  it  is  below  minimal  standards 

5  =  it  is  far  bellow  minimal  standards 

3.  In  your  estimate,  what  is  the  percentage  of  RPs  that  underperform  and  what  is  the 
percentage  or  RPs  that  excel  in  instrument  approach  procedures? 

4.  What  is  your  recommendation  when  you  notice  a  RP  is  having  problems  with 
instrument  approach  procedures  -  what  training  approach  do  you  recommend  a  RP 
take  in  terms  of  the  skill  remediation? 

5.  Aside  from  the  answers  provided  from  the  previous  question,  list  any  additional 
information  or  comments  that  relates  to  trends  associated  with  instrument  approach 
procedures. 

6.  What  supplemental  instructional  techniques  (ones  that  are  not  directly  required  by 
the  Training  &  Readiness  (T&R)  manual)  are  used  in  your  squadron  in  order  to 
better  prepare  RPs  for  instrument  approach  procedures? 

7.  In  your  opinion,  what  is  the  most  effective  supplemental  technique  for  preparing 
RPs  for  instrument  approach  procedures? 

8.  What  element  of  that  supplemental  technique  do  you  think  is  very  effective  for 
preparing  RPs  for  instrument  approach  procedures? 

9.  In  your  opinion,  how  often  should  RPs  ideally  use  that  type  of  supplemental 
technique? 

10.  In  your  opinion,  is  there  any  additional  training  solution  that  should  be  made 
available  to  the  RPs  to  better  prepare  them  for  instrument  approach  procedures  in 
the  actual  aircraft? 

11.  What  major  learning  and  training  objectives  should  that  system  support? 


37 


2. 


CONSENT 


Naval  Postgraduate  School 
Consent  to  Participate  in  Research 

AUTHORITY:  10  U.S.C.  5013;  10  U.S.C.  5041;  10  U.S.C.  1074f;  32  CFR  64.4;  DoD  Instruction  1215.13;  DoO  Instruction  3001.02;  C1CSM 
3150.13C;  DoD  Instruction  6490.03;  SECNAVINST  1770.3D;  MCO  7220.50B;  and  E.O.  9397  (SSN),  as  amended. 

Introduction.  You  are  invited  to  participate  in  a  research  study  entitled  Proof-of-concept  Part-task  Trainer  to  Enhance  Situational 
Awareness  for  Instrument  Approach  Procedures  in  Aviation  Domain.  The  purpose  of  this  research  study  is  to  investigate  important 
issues  novice  pilots  have  flying  instrument  approach  procedures. 

Procedures.  You  will  be  asked  to  answer  several  questions  utilizing  an  online  survey  tool  called  LimeSurvey.  The  survey  is  focused  on 
your  experiences  and  understandings  of  current  instrument  approach  procedure  techniques.  The  surveys  should  take  approximately  30 
minutes  to  complete. 

Location.  The  survey  will  be  conducted  online. 

Cost.  There  is  no  cost  to  participate  in  this  research  study. 

Voluntary  Nature  of  the  Study.  Your  participation  in  this  study  is  strictly  voluntary.  It  is  important  to  know  that  if  you  choose  to 
participate  you  can  change  your  mind  at  any  time  and  withdraw  from  the  study.  You  will  not  be  penalized  in  any  way  or  lose  any 
benefits  to  which  you  would  otherwise  be  entitled  if  you  choose  not  to  participate  in  this  study  or  to  withdraw.  The  alternative  to 
participating  in  the  research  is  to  not  participate  in  the  research. 

Potential  Risks  and  Discomforts.  There  is  a  minimal  risk  of  breach  of  confidentiality  and  the  survey  process  does  not  involve  greater 
than  minimal  risk. 

Anticipated  Benefits.  Anticipated  benefits  from  this  study  are  for  the  Marine  Corps  and  any  other  services  that  conduct  instrument 
approach  procedures  (IAP).  Data  collected  will  help  in  identifying  methods  to  effectively  use  virtual  environments  for  the  conduct  of  a 
part  task  trainer  within  the  IAP  mission  set.  You  will  not  directly  benefit  from  your  participation  in  this  research. 

Compensation  for  Participation.  No  tangible  compensation  will  be  given. 

Confidentiality  &  Privacy  Act.  Any  information  that  is  obtained  during  this  study  will  be  kept  confidential  to  the  full  extent  permitted  by 
law.  Information  collected  will  be  removed  from  LimeSurvey  and  stored  by  the  Principle  Investigator.  All  efforts,  within  reason,  will  be 
made  to  keep  your  personal  information  in  your  research  record  confidential,  but  total  confidentiality  cannot  be  guaranteed.  No 
information  will  be  publicly  accessible  that  could  identify  you  as  a  participant.  Understand  that  records  of  your  participation  will  be 
maintained  by  NPS  for  ten  years. 

Points  of  Contact.  If  you  have  any  questions  or  comments  about  the  research,  or  you  experience  an  injury  or  have  questions  about  any 
discomforts  that  you  experience  while  taking  part  in  this  study  please  contact  the  Principal  Investigator,  Dr.  Amela  Sadagic,  (831)  656- 
3819,  asadagic@mps.edu.  Questions  about  your  rights  as  a  research  subject  or  any  other  concerns  may  be  addressed  to  the  Navy 
Postgraduate  School  IRB  Chair,  Dr.  Larry  Shattuck,  lgshattu@nps.edu. 

Statement  of  Consent.  I  have  read  the  information  provided  above.  I  have  been  given  the  opportunity  to  ask  questions  and  all  the 
questions  have  been  answered  to  my  satisfaction.  I  have  been  provided  a  copy  of  this  form  for  my  records  and  I  agree  to  participate  in 
this  study.  I  understand  that  by  agreeing  to  participate  in  this  research  and  signing  this  form,  1  do  not  waive  any  of  my  legal  rights. 

I  consent  to  participate  in  the  research. 


Yes  No 
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3. 


IP 


IP  Survey 


1.  Does  a  training  gap  exist  between  the  classroom/computer-based  trainers  and  the  simulator? 
Choose  one  of  the  following  answers 

Yes  Please  enter  your  comment  here: 

No 


2.  Does  a  training  gap  exist  between  the  simulator  and  the  aircraft  during  the  familiarization  (FAM)  stage? 
Choose  one  of  the  following  answers 

Yes  Please  enter  your  comment  here; _ 

No 


3.  During  the  first  flight  event  in  the  FAM  stage,  most  replacement  pilots  (RPs)  are  able  to  conduct  all  procedures  and  communications 
of  instrument  approach  (IA)  at  an  acceptable  standard  per  the  grade  sheet. 

Choose  one  of  the  following  answers 

Strongly  Agree 
Agree 

Somewhat  Agree 
Neither  Agree  nor  Disagree 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


4.  In  your  opinion,  what  are  the  top  three  difficulties  the  RPs  have  during  FAM  flights?  (e.g.  -  Flows,  Checklists,  Getting  behind  the 
aircraft,  comms,  etc.) 


5.  List  any  additional  information  or  comment  that  relates  to  trends  associated  with  the  performance  of  RPs  in  FAM  stage. 
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6.  Which  of  the  following  supplemental  instructional  techniques  that  are  not  directly  required  by  the  Training  &  Readiness  (T&R) 
manual,  are  used  in  the  squadron  in  order  to  better  prepare  RPs  for  the  FAM  stage?  Check  any  that  applies,  and  select  frequency  with 
which  it  is  used. 


Daily  Weekly  Monthly 

Chair  Flying 
Formal  quizzes/tests 

Self  study  of  supplemental 
readings 


7.  In  your  opinion,  what  is  the  effectiveness  of  each  supplemental  technique  used  to  prepare  RPs  for  FAM  flights? 

Neither 

Very  Somewhat  Effective  nor  Somewhat 

Effective  Effective  Effective  Ineffective  Ineffective  Ineffective 

Chair  Flying 
Formal  Quizzes/tests 

Self  Study  of  Supplemental 
Readings 


8.  In  your  opinion,  how  often  should  RPs  ideally  participate  in  chair  flying  sessions  while  preparing  for  the  FAM  stage? 
Choose  one  of  the  following  answers 

Daily 

Weekly 

Monthly 

Other: 

9.  List  three  most  significant  reasons  why  you  recommend  using  chair  flying  to  RPs  under  your  instruction: 


10.  In  your  opinion,  what  are  the  limitations  of  chair  flying? 

Check  any  that  apply 

Lack  of  IP  availability  to  lead  evolution. 

Lack  of  IP  availability  to  create  new  scenarios/scripts. 

Time  constraints  for  all  participating  aircrew  (Crew  Day,  Crew  Rest,  etc). 
The  inability  to  train  large  groups  at  one  time. 

Other: 


Rarely 


Very 

Ineffective 


40 


11.  RPs  who  perform  well  on  formal  quizzes/tests  more  likely  to  perform  well  in  the  aircraft  during  FAM  flights? 
Choose  one  of  the  following  answers 

Strongly  Agree 
Agree 

Somewhat  Agree 
Neither  Agree  nor  Disagree 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 

12.  List  the  three  most  significant  reasons  why  you  think  RPs  should  take  formal  quizzes/tests. 


1. 

2. 

3. 


13.  Describe  other  supplemental  instructional  techniques  that  you  use  or  think  those  could  help  RPs  become  better  prepared  to  conduct 
their  first  IAPs.  This  may  include  things  that  RPs  do  on  there  own  time  without  direct  involvement  of  an  IP. 

1. 

2. 

3. 


14.  Imagine  you  are  tasked  with  designing  a  new  tool  to  help  prepare  RPs  for  their  first  flight  involving  IAPs  in  a  flight  simulator.  The 
system  could,  for  example,  allow  visualization  of  IAP  and  practicing  of  a  specific  skill  set  associated  with  instrument  approach.  What 
major  training  objectives  should  that  system  support? 


1. 

2. 

3. 


15.  The  features  of  that  system  may  allow  an  IP  to  define  details  of  some  scenario  and  a  set  of  popup  questions  that  could  be  presented 
to  a  RP  at  various  checkpoints  or  phases  of  the  flight  to  quiz  the  RP's  knowledge.  What  other  system  capabilities  would  you  recommend 
that  such  training  system  could  have? 

Check  any  that  apply 

Ability  to  question  the  knowledge  about  in-range  and  approach  checklists 

Ability  to  have  text  or  voice  operated  radio  calls 

Ability  to  select  from  a  set  of  prebuilt  airfields  with  their  approaches 

Other: 


16.  In  order  to  test  RP's  situational  awareness  in  the  training  system  described  in  previous  question,  what  element  of  RP's  situational 
awareness  would  you  test  and  how  would  you  go  about  it  in  that  training  system? 


17.  What  would  be  the  best  hardware  platform  to  support  desired  functionality  of  this  training  system? 
Choose  one  of  the  following  answers 

Computer  with  wall  size  projection  display 

Computer  with  large  monitor 

Laptop  Computer 

Tablet  with  touch  screen 

Smart  Phone 

18.  What  instructor  billets  do  you  currently  hold  in  the  squadron? 


1. 

2. 

3. 

4. 

5. 

19.  How  many  years  of  experience  do  you  have  as  a  flight  instructor? 
Only  numbers  may  be  entered  in  this  field. 
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4, 


RP 


RP  Survey 


1.  Have  you  conducted  your  first  flight  event  in  the  familiarization  (FAM)  stage  of  the  Training  and  Readiness  manual,  and  if  you  did, 
when  did  that  occur?  Please  enter  date  in  the  comment  box. 

Choose  one  of  the  following  answers 

Yes  Please  enter  your  comment  here: _ 

No 


2.  In  your  opinion,  do  you  have  enough  opportunities  to  practice  instrument  approaches  (IAPs)  to  stay  proficient? 

Yes  No 

3.  During  the  first  flight  event  in  the  FAM  stage,  I  was  able  to  conduct  all  IAP  procedures  and  communications  at  an  acceptable  standard 
per  the  grade  sheet. 

Choose  one  of  the  following  answers 

Strongly  Agree 
Agree 

Somewhat  Agree 
Neither  Agree  nor  Disagree 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


4.  In  your  opinion,  what  are  the  top  three  difficulties  you  had  during  these  flights?  (e.g.  -  Flows,  Checklists,  Getting  behind  the  aircraft, 
comms,  etc.) 


1. 


3. 


5.  List  any  additional  information  or  comments  that  relates  to  trends  associated  with  your  performance  during  FAM  stage. 


1. 

2. 

3. 
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6.  Which  of  the  following  supplemental  instructional  techniques  that  are  not  directly  required  by  the  Training  &  Readiness  (T&R) 
manual,  are  used  in  the  squadron  in  order  to  better  prepare  you  for  the  FAM  stage?  Check  any  that  applies,  and  select  frequency  with 
which  it  is  used. 


Daily 


Weekly 


Monthly 


Chair  Flying 
Formal  quizzes/tests 

Self  study  of  supplemental 
readings 


7.  In  your  opinion,  what  is  the  effectiveness  of  each  supplemental  technique  used  to  prepare  you  for  FAM  flights? 


Neither 

Very  Somewhat  Effective  nor  Somewhat 

Effective  Effective  Effective  Ineffective  Ineffective  Ineffective 

Chair  Flying 
Formal  Quizzes/tests 

Self  Study  of  supplemental 
Readings 

Other 


8.  In  your  opinion,  how  often  should  you  ideally  participate  in  chair  flying  sessions  while  preparing  for  the  FAM  stage? 
Choose  one  of  the  following  answers 

Daily 

Weekly 

Monthly 

Other: 

9.  List  the  most  significant  reasons  why  you  use  chair  flying. 


1. 

2. 

3. 

10.  In  your  opinion,  what  are  the  limitations  of  chair  flying?  Check  all  that  apply: 
Check  any  that  apply 

Lack  of  IP  availability  to  lead  evolution 

Lack  of  IP  availability  to  create  new  scenarios/scripts 

Time  constraints  for  all  participating  aircrew  (Crew  Day,  Crew  Rest,  etc) 

The  inability  to  train  large  groups  at  one  time 
Other: 


Very 

Ineffective 
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11.  RPs  who  perform  well  on  formal  quizzes/tests  more  likely  to  perform  well  in  the  aircraft  during  FAM  flights? 
Choose  one  of  the  following  answers 

Strongly  Agree 
Agree 

Somewhat  Agree 
Neither  Agree  nor  Disagree 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 

12.  List  three  significant  reasons  why  you  think  RPs  should  take  formal  quizzes/tests. 

1. 

2. 

3. 


13.  Describe  other  supplemental  instructional  techniques  that  you  use  or  think  those  could  help  RPs  become  better  prepared  to  conduct 
their  first  IAPs.  This  may  include  things  you  do  on  your  own  time  without  direct  involvement  of  an  IP. 


14.  Imagine  you  are  tasked  with  designing  a  new  tool  to  help  prepare  RPs  for  their  first  flight  involving  IAPs  in  a  flight  simulator.  The 
system  could,  for  example,  allow  visualization  of  IAP  and  practicing  of  a  specific  skill  set  associated  with  instrument  approach.  What 
major  training  objectives  should  that  system  support? 


1. 

2. 


3. 


15.  The  system  may  allow  one  to  define  details  of  some  scenario  and  a  set  of  popup  questions  that  could  be  presented  to  a  RP  at  various 
checkpoints  or  phases  of  the  Tight  to  quiz  the  RP's  knowledge.  What  other  system  capabilities  would  you  recommend  that  such  training 
system  could  have? 

Check  any  that  apply 

Ability  to  question  the  knowledge  about  in-range  and  approach  checklists 

Ability  to  have  text  or  voice  operated  radio  calls 

Ability  to  select  from  a  set  of  prebuilt  airfields  with  their  approaches 

Other: 


5.  DEMONSTRATION  QUESTIONS 

What  functionality  is  currently  missing  from  the  prototype? 

Which  functionality  of  demoed  prototype  worked? 

Which  functionality  of  demoed  prototype  did  not  work?  What  was  missing? 
Which  functionality  of  demoed  prototype  should  be  refined  (corrected)? 
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APPENDIX  B:  TASK  ANALYSIS 


1.  NATOPS  TASK  DECOMPOSITION 

TA  is  from  the  Department  of  the  Navy’s  (2014)  Naval  Air  Training  and  Operating 
Procedures  Standardization  Flight  Manual. 

a.  PF  In-Range  Flows 


Figure  11.  PF  In-Range  Flows 


1.  PRESSURIZATION  —  Set. 

a.  Pressure  Mode  Select  —  Set. 

b.  Rate  Selector  —  Set. 

c.  LND/CONST  —  Set. 

d.  PM  states  the  settings  of  the  pressure  mode  select  switch,  rate  selector  knob  and 

LDG/CONST  altitude. 

2.  Anti-Ice/De-ice  —  As  required. 

a.  WING/EMP  switch  ON,  ANTI-ICE  (as  required,  if  icing  is  anticipated). 

CAUTION 

The  wing/empennage  anti-ice/de-ice  system  should  not  be  operated  in  the  anti-icing 
mode  except  during  approach  and  landing.  Operation  below  -30  degrees  Celsius  at  power 
settings  above  2,500  HP,  or  above  20,000  feet  pressure  altitude  at  any  power  setting  may 
degrade  engine  life. 
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Note 

•  Engine  anti-ice  should  be  ON  if  SAT  is  less  than  or  equal  to  10  degrees  Celsius 
with  visible  moisture. 

•  Wing/empennage  anti-icing  in  anti-ice  mode  should  be  turned  on  in  sufficient 
time  prior  to  landing  to  ensure  the  vertical  stabilizer  is  completely  anti-iced.  This 
may  take  as  long  as  2.5  minutes.  Failure  to  do  so  will  cause  the  X-WIND 
LIMITED  ACAWS  message  to  be  displayed  when  the  landing  gear  are  lowered, 
and  the  side  slip  warning  system  will  limit  crosswind  landing  capability  if  icing  is 
detected. 

•  Failure  to  turn  on  wing/empennage  anti-icing  in  anti-ice  mode  prior  to  lowering 
the  landing  gear  when  icing  is  detected  causes  the  WING/EMP  A/I  NOT  ON 
ACAWS  message  to  be  displayed. 

3.  HUD  —  Set  as  necessary  for  the  approach. 

4.  CNBP  —  Set. 

a.  NAVAIDs. 

(1)  Set  for  required  approach  NAVAID  guidance  —  (as  required). 

(2)  NAVAIDs  —  Set  as  required. 

Note 

•  Ensure  that  NAVAIDs  are  tuned,  identified  and  selected  and  that  the  instruments 
and  radar  are  set  for  the  approach. 

•  The  standard  NAVAID  Tuning  procedure  is  to  use  the  station  identification 
letters.  After  tuning  the  station,  verify  the  frequency. 

5.  AMU  —  Set. 

a.  HDDs  —  Set  to  desired  approach  configuration. 

b.  NAV  SELECT. 

(1)  CDI  SOURCE  —  Select  the  approach  NAVAID. 

(2)  PNTR  1/PNTR  2  —  Set  as  required. 

(3)  COURSE  knob  —  Set. 

(4)  CDI  SOURCE  —  Set. 

Note 

•  Set  the  planned  final  approach  course  if  able. 

•  If  the  autopilot  is  engaged  on  the  INAV  controlling  solution  side  and  the 

•  CDI  source  is  changed  to  set  up  for  the  approach,  the  NAV  mode  will  disengage. 
Once  the  course  is  set  for  the  VOR  or  TACAN  approach,  the  CDI  source  can  be 
returned  to  the  INAV  solution.  Ensure  NAV  mode  is  reengaged  or  heading  mode 
is  selected  if  INAV  is  reselected  as  the  CDI  source. 

•  If  changing  CDI  source  in  order  to  set  the  course  for  an  approach,  ensure  the  CDI 
source  is  reset  to  INAV  and  the  flight  director  is  reengaged  if  applicable. 
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c.  NAV  RADAR  —  Select  overlays. 

d.  Digital  Map  Display  —  As  required. 

e.  EMS  EVENT  RECORD  —  Press. 

Note 

EMS  EVENT  RECORD  can  be  pressed  anytime  level  flight  and  constant  airspeed  can  be 
maintained  for  30  seconds.  EMS  EVENT  RECORD  only  needs  to  be  pressed  once  per 
flight  to  record  propeller  balance  data. 

6.  REF/MODE  panel  —  Set. 

a.  RAD  ALT  —  Set. 

(1)  Precision  Approach  —  50  feet  below  HAT. 

(2)  Non-Precision  Approach  —  250  feet. 

(3)  Circling/Visual  Approach  —  300  feet. 

(4)  VFR  Pattern/Tactical  —  Set  (As  required). 

b.  FPA  —  Set. 

c.  MINS  —  Set. 

(1)  IFR  Approach  —  Set  published  minimums. 

(2)  VFR  Approach  —  Set  (As  required). 

d.  BARO  SET  —  Set. 

7.  CNI-MU  —  Set. 

a.  ROUTE/LEGS 

WARNING 

Terminal  area  waypoints  and  fixes  shall  be  identified  by  reference  to  the  appropriate 
navigational  aid  as  published  in  the  approved  approach  procedure. 

Note 

Approach  construction  in  the  CNI  should  be  considered  as  a  secondary  priority  to  the 
accurate  programming  of  flow  elements  that  precede  it.  If  time  permits,  construct  the 
approach  in  the  CNI-MU  to  represent  the  approach  as  closely  as  possible. 

b.  TOLD  INIT. 

(1)  TOLD  INIT  entries. 

(2)  Check  Landing  Speeds. 

(3)  Check  Landing  Distances. 

CAUTION 

Ensure  the  cargo  weight  used  by  the  CNI-MU  for  landing  data  is  current.  If  an  airdrop 
has  been  performed  or  cargo  jettisoned,  input  the  new  weight  so  that  the  system  can 
calculate  accurate  landing  data. 


Note 

If  landing  data  is  manually  calculated,  new  landing  data  will  be  required  for  aircraft 
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weight  change  greater  than  5,000  pounds,  pressure  altitude  change  greater  than  1,000  feet 
or  outside  air  temperature  change  greater  than  5  degrees  Celsius. 

c.  MC  INDEX. 

(1)  V  SPEEDS. 

(2)  TACTICAL  PLOT. 

d.  INDEX. 

8.  Belts/Harnesses  —  Fastened. 

9.  Approach  Briefing  —  Complete. 

a.  Reference  Chapter  16  for  the  arrival-briefing  guide. 

2.  Approach  Brief  Example 

“We  will  fly  radar  vectors  to  the  ILS  RWY  17  at  Navy  Fort  Worth.  The  weather  is  800 
with  2  miles  visibility,  the  required  weather  is  200  and  3/4.  The  field  elevation  is  650, 
650  is  set  in  the  pressurization  panel.  The  NAVAIDs  are  set  with  I-NFW  in  the  VORs 
and  NFW  in  the  TACANs.  I  will  fly  the  approach  with  VOR1  and  the  CDI  course  is  set 
to  174.  DH  is  836,  840  is  set  in  MINS.  RAD  ALT  is  set  150.  Speeds  for  a  100%  flap 
landing  are  135,  125,  118.  There  is  an  LZ  in  the  CNI.  Stable  approach  gate  is  1650. 1  will 
fly  automated  with  autothrottles.  There  is  no  significant  terrain,  but  there  are  numerous 
towers  along  the  approach  corridor.  The  published  missed  approach  instructions  are 
climb  heading  174  to  3000,  intercept  the  FUZ  218  radial  to  MOLKE  and  hold.” 

3.  IN-RANGE  CHECKLIST 

Trigger:  At  the  completion  of  the  Crew  Briefing  the  PF  should  call  for  the  checklist. 

1 .  Altimeters:  “Set _ ”  (CP). 

“Set _ twice”  (P). 

WARNING 

Altimeters  will  be  set  to  station  pressure  (QNH/QFE)  if  available  when  transiting  the 
transition  level.  Altimeters  may  be  set  when  above,  but  cleared  through,  the  transition 
level.  The  In-Range  Checklist  shall  not  be  called  complete  until  the  QNH/QFE  has  been 
set. 


Note 

The  standby  altimeter  shall  be  set  separately. 

2.  Pressurization:  “Set, _ , _ , _ ”  (PM). 

a.  PM  states  the  settings  of  the  pressure  mode  select  switch,  rate  selector  knob  and 
LDG/CONST  altitude. 

3.  ANTI-ICE/DE-ICE  switch:  “ _ ”  (PM). 

a.  PM  states  the  setting. 
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4.  ACAWS:  “Checked”  (PM). 

5.  Lights:  “Set”  (PM). 


Note 

Consideration  should  be  given  to  using  wing  tip  taxi  lights,  formation  lights  and  pod 
illumination  lights  when  descending  into  high-density  traffic  areas. 

6.  TCAS:  “ _ , _ ”  (PM). 

a.  TA/RA  —  As  required. 

b.  ABV/Below/Normal  —  Check. 

c.  PM  states  TCAS  mode  setting. 

7.  Belts/Harnesses:  “Fastened”  (PM). 

8.  Passengers  and  Cargo:  “Secured”  (LM/CC/CM). 

9.  In-Range  Checklist:  “Complete”  (PM). 

4.  APPROACH  CHECKLIST 

Trigger:  When  the  final  approach  navaid  is  set  in  the  CDI  source  of  the  AMU  NAV 
SELECT  page. 


Note 

The  APPROACH  CHECKLIST  should  only  be  used  when: 

•  NAVAIDs  have  been  tuned  and  identified,  if  applicable,  and 

•  Below  TA/TL,  and 

•  Entering  the  terminal  environment. 

1.  Altimeters:  “Set _ ”  (CP). 

“Set _ twice”  (P). 

a.  Whenever  the  altimeters  are  reset,  both  pilots  state  the  new  setting. 

2.  NAVAIDs:  “Set _ ”  (PM). 

a.  The  PM  states  the  primary  approach  NAVAID  Frequency. 

3.  CDI  Source:  “ _ , _ ”  (PM,  PF). 

a.  CDI  source  set  for  approach  NAVAID. 

b.  Course  Select  Knob  —  Set. 

c.  Both  pilots  state  the  CDI  source  selected  and  CDI  course. 

4.  Rad  Alt:  “ _ Set”  (PM). 

a.  The  PM  states  the  radar  altimeter  set  in  the  REF/SET  Mode  Select  panel. 


49 


5.  Minimums:  _ _Set”  (PM). 

a.  The  PM  states  the  approach  minimums  set  in  the  REF/SET  Mode  Select  panel. 

6.  FUEL  MANAGEMENT  panel:  _ ”  (PM/ACM). 

a.  The  PM  or  the  ACM  turns  the  SPR  DRAIN  switch  ON,  if  required,  and  verifies 
the  fuel  panel  configuration. 

b.  Fuel  quantity  —  Check. 

c.  Distribution  —  Check. 

d.  State  the  fuel  quantity,  sink  rate  limit,  and  SPR  drained. 

Note 

•  Do  not  transfer  fuel  from  tank  to  tank  during  landings  or  touch  and  go  landings. 

•  Refer  to  Fuel  Management,  Approach  and  Landing  procedures  for  allowable  fuel 
system  configuration. 

7.  Briefing:  “Complete”  (PF). 


Note 

Refer  to  Chapter  16  for  the  Arrival  Briefing. 

8.  Approach  Checklist:  “Complete”  (PM). 

5.  BEFORE  LANDING  FLOW 

The  pilot  may  direct  lowering  of  the  flaps  and  gear  before  calling  for  the  BEFORE 
LANDING  Checklist. 


Note 

•  Wing/empennage  anti-icing  in  anti-ice  mode  should  be  turned  on  in  sufficient 
time  prior  to  landing  to  ensure  the  vertical  stabilizer  is  completely  anti-iced.  This 
may  take  as  long  as  2.5  minutes.  Failure  to  do  so  will  cause  the  X-WIND 
LIMITED  ACAWS  message  to  be  displayed  when  the  landing  gear  are  lowered, 
and  the  side  slip  warning  system  will  limit  crosswind  landing  capability  if  icing  is 
detected. 

•  Failure  to  turn  on  wing/empennage  anti-icing  in  anti-ice  mode  prior  to  lowering 
the  landing  gear  when  icing  is  detected  causes  the  WING/EMP  A/I  NOT  ON 
ACAWS  message  to  be  displayed. 

1.  FLAPS:  “FLAPS  50”  (PF). 


CAUTION 

Changing  the  direction  of  flaps  travel  while  the  flaps  are  moving  may  damage  the  flaps 
motor. 


a.  The  PM  selects  the  flap  handle  to  50%  and  reports  when  the  flaps  are  at  50%. 
“Flaps  are  50”  (PM). 
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Note 

Normal  landing  configuration  is  with  100  percent  flaps  and  may  be  selected  after  the  gear 
is  reported  down  and  prior  to  crossing  the  threshold. 

2.  Gear:  “Gear  Down  Before  Landing  Checklist”  (PF). 

3.  Lights:  Extended  and  On  (PM). 

a.  Landing  lights  —  Extended  and  On  (As  required). 

b.  TAXI  LIGHTS  switch  —  ON. 

c.  WINGTIP  TAXI  LIGHTS  switch  —  As  required. 

CAUTION 

Extending  landing  lights  while  illuminated  may  induce  pilot  vertigo  when  in  IMC 
conditions. 


Note 

The  Pilot  Monitoring  should  operate  the  gear  handle. 

6.  BEFORE  LANDING  CHECKLIST 

1.  FLAPS:  “FLAPS  50”  (PM). 

2.  GEAR:  “3  Down”  (PM). 

“3  Down,  Centered”  (PF). 


CAUTION 

Check  the  nosewheel  steering  indicator  to  ensure  the  nosewheel  is  centered. 

Note 

Ensure  anti-skid  on  and  no  anti-skid  ACAWS  messages  displayed. 

3.  Lights:  _ ”  (PM). 

a.  PM  states  whether  the  landing  lights  are  extended  or  retracted  and  On  or  Off. 

4.  Before  Landing  Checklist:  “Complete”  (PM). 
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APPENDIX  C:  RESEARCH  APPROVALS 


1.  NPS IRB 


Naval  Postgraduate  School 

Human  Research  Protection  Program 


From:  President,  Naval  Postgraduate  School  (NPS) 

To:  Dr.  Amela  Sadagic,  Modeling,  Virtual  Environments  and 

Simulation  Institute  (MOVES) 

LCDR  Lee  Sciarini,  USN 
Capt  Nicholas  Arthur,  USMC 
Via:  Vice  Chair,  Institutional  Review  Board  (IRB) 

Sub  j :  PROOF-O F-CONCEPT  PART-TASK  TRAINER  TO  ENHANCE  SITUATIONAL 
AWARENESS  FOR  INSTRUMENT  APPROACH  PROCEDURES  IN  AVIATION 
DOMAIN 

Enel:  (1)  Approved  IRB  Protocol  Amendment 

1.  The  NPS  IRB  is  pleased  to  inform  you  that  the  NPS  President 
has  approved  your  protocol  amendment  (NPS  IRB#  NPS . 2017 . 0029- 
AM01-EP7-A) .  The  approved  IRB  Protocol  is  found  in  enclosure 
(1).  Completion  of  the  CITI  Research  Ethics  Training  has  been 
confirmed . 

2.  This  approval  expires  on  5  March  2018.  If  additional  time 
is  required  to  complete  the  research,  a  continuing  review  report 
must  be  approved  by  the  IRB  and  NPS  President  prior  to  the 
expiration  of  approval.  At  expiration  all  research  (subject 
recruitment,  data  collection,  analysis  of  data  containing  PII) 
must  cease. 

3.  You  are  required  to  obtain  consent  according  to  the 
procedure  provided  in  the  approved  protocol. 

4.  You  are  required  to  report  to  the  IRB  any  unanticipated 
problems  or  serious  adverse  events  to  the  NPS  IRB  within  24 
hours  of  the  occurrence. 

5.  Any  proposed  changes  in  IRB  approved  research  must  be 
reviewed  and  approved  by  the  NPS  IRB  and  NPS  President  prior  to 
implementation  except  where  necessary  to  eliminate  apparent 
immediate  hazards  to  research  participants  and  subjects. 

6.  As  the  Principal  Investigator  (PI)  it  is  your  responsibility 
to  ensure  that  the  research  and  the  actions  of  all  project 
personnel  involved  in  conducting  this  study  will  conform  with 
the  IRB  approved  protocol  and  IRB  requirements/policies. 
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7.  At  completion  of  the  research,  no  later  than  expiration  of 
approval,  the  PI  will  close  the  protocol  by  submitting  an  End  of 
Experiment  Report. 


K.  J.  Euske 


DM  rr-*  J  (vm< 

Ou'&VSW 

C*US 

Dm*  airmn  'Annu-cnx 


Kenneth  J.  Euske,  PhD 
Vice  Chair 

Institutional  Review  Board 


Ronald  A.  Route 

Vice  Admiral,  U.S.  Navy  (Ret.) 

President,  Naval  Postgraduate  School 


Date 


MAR  3  0  2017 


54 


2. 


DEPUTY  COMMANDANT  OF  AVIATION 


DEPARTMENT  OF  THE  NAVY 

HEADQUARTERS  UNITED  STATES  KARINE  CORPS 
3000  MARINE  CORPS  PENTAGON 
WASHINGTON,  DC  20350-3000 

■MUrLYUfEKIO 

3970 
AVN 

6  Apr  17 

From:  Deputy  Commandant  for  Aviation 

To:  Captain  Nicholas  L.  Arthur  1285751540/7557  USMC 

Sub j :  RESEARCH  SPONSORSHIP  FOR  CAPTAIN  NICHOLAS  L.  ARTHUR 
1285751540/7557  USMC 

1.  I  support  the  research  proposed  by  Captain  Nicholas  Arthur 
regarding  the  "Proof-of-concept  Part-task  Trainer  to  Enhance 
Situational  Awareness  for  Instrument  Approach  Procedures  in  Aviation 
Domain"  within  the  broader  research  area  of  Modeling  and  Simulations. 

2.  The  part-task  trainer  can  be  used  as  a  bridge  between  basic  chair 
flying  and  actually  flying  approaches  in  a  full-motion  simulator  or 
even  the  aircraft.  This  proof  of  concept  will  increase  the  likelihood 
that  the  pj.lot  will  be  capable  of  successfully  flying  the  Approach 
while  conducting  additional  tasks  that  often  occur. 

3.  The  research  will  focus  on  the  following  questions: 

a.  What  is  the  feasibility  of  developing  a  prototype  virtual 
reality  (VR)  system  that  supports  the  training  of  instrument  approach 
procedures  and  situational  awareness? 

b.  Are  there  training  gaps  identified  in  connection  with 
procedures  and  situational  awareness  needed  for  conducting  an 
instrument  approach  by  the  pilots? 

c.  On  what  cues  (cockpit,  environmental,  CRM,  ATC,  haptic,  etc.) 
should  KC-130J  FRD  students  focus  in  support  of  maintaining 
situational  awareness  during  instrument  approaches? 

4.  This  research  will  support  the  use  of  modeling  and  simulations  for 
the  Aviation  community  and  provide  cost  effective  training 
enhancements  for  pilots. 

5.  Colonel  Todd  Lyons,  Associate  Dean  of  Research  at  the  Naval 
Postgraduate  School,  can  address  research  specific  topics  at 
831-884-8317  or  twlyons@nps.edu. 


J.  M.  DAVIS 
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3. 


USMC  HUMAN  RESEARCH  PROTECTION 


DEPARTMENT  OF  THE  NAVY 

HEADQUARTERS  UNITED  STATES  MARINE  CORPS 
3300  RUSSELL  ROAD 
QUANTICO,  VA  22134-5001 

3900 
C  4623 
7  Apr  17 


From:  Human  Research  Protection  Official,  U.S.  Marine  Corps  (Attention:  Ms  Leah 

Watson),  1019  Elliot  Rd,  Quantico,  VA  22134 

To:  Dr.  Amelia  Sadagic,  Modeling,  Virtual  Environments  and  Simulation 

Institute  (MOVES),  Faculty  Advisor 

Lieutenant  Commander  Lee  Sciarini,  USN,  Faculty  Advisor,  Naval  Post 
Graduate  School,  Monterey,  CA 

Captain  Nicholas  Arthur,  USMC,  Student  Investigator,  Naval  Post 
Graduate  School,  Monterey,  CA 

Subj:  HUMAN  RESEARCH  PROTECTION  PROGRAM  U.S.  MARINE  CORPS 

ADMINISTRATIVE  17  REVIEW  OF  PROPOSED  STUDY:  “PROOF-OF- 
CONCEPT  PART-TASK  TRAINER  TO  ENHANCE  STITUATIONAL 
AWARENESS  FOR  INSTRUMENT  APPROACH  PROCEDURES  IN 
AVIATION  DOMAIN” 

Ref:  (a)  DoDI  3216.02 

(b)  SECNAVINST  3900. 16D 

(c)  MCO  3900.18 

(d)  U.S.  Marine  Corps  Human  Research  Protection  Program  (HRPP)  Policy 
and  Procedures  (27  Sep  16) 

(e)  DOD  N-40042  Naval  Postgraduate  School  DOD-Navy  Assurance 

(f)  MCO  5300.18  Marine  Corps  Survey  Program 

Enel:  (1)  NPS.2017.0029-IR-EP7-A 

(2)  NPS  .2017 .0029- AMO  1 -EP7 -A 

(3)  DC  Aviation  ltr  3970  AVN  of  6  Apr  7 

1.  Per  references  (a)  through  (d),  I  have  performed  an  Administrative  Review,  on  behalf 
of  the  Marine  Corps,  of  the  research  titled  “Proof-of-Concept  Part-Task  Trainer  to 
Enhance  Situational  Awareness  for  Instrument  Approach  Procedures  in  Aviation 
Domain.”  The  objective  of  this  research  is  to  explore  the  feasibility  of  developing  a 
prototype  virtual  reality  (VR)  system  that  supports  the  training  of  instrument  approach 
procedures  and  situational  awareness.  This  research  is  being  conducted  as  partial 
fulfillment  of  academic  requirements  for  Captain  Arthur  as  a  student  at  the  Naval 
Postgraduate  School  (NPS). 
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2.  The  research  plan  at  enclosure  (1)  and  the  first  amendment  at  enclosure  (2),  as 
submitted  to  the  NPS  IRB,  has  been  reviewed  by  this  office  to  ensure  compliance  with 
requirements  the  U.S.  Marine  Corps  Human  Research  Protection  Program. 


3.  As  outlined  in  the  research  plan  at  enclosure  (1),  the  research  will  involve  voluntary 
subject  participation  in  an  online  survey  about  their  understanding  of  current  instrument 
approach  procedure  training  techniques.  The  results  will  be  used  by  the  researcher  in 
determining  the  prototype  interface  design.  Enclosure  (1)  also  includes  the  Commanding 
Officer  approval  for  solicitation  of  subjects,  from  the  Commanding  Officers  of  VMGR- 
152,  VMGR-252,  and  VMGR-352.  Enclosure  (3),  from  the  Deputy  Commandant  for 
Aviation,  provides  the  General  Officer  level  support  required  for  research  being 
conducted  by  institutions  other  than  the  Marine  Corps. 

4.  Reference  (e)  is  the  DOD  Institutional  Assurance  for  Naval  Postgraduate  School  which 
expires  30  November  2019.  As  reflected  at  enclosures  (1),  the 

NPS  IRB  reviewed  and  approved  your  research  proposal  as  meeting  criteria  for 
Expedited  Review  under  Categories  7  and  approved  waiver  of  the  requirement  for 
documentation  of  informed  consent  under  32  CFR  219.1 17((2). 

5.  Review  by  the  NPS  IRB  and  the  DC  Aviation  approval  satisfy  Marine  Corps  review 
requirements  for  this  study.  Approval  by  the  NPS  IRB  assures  that  the  members  of  the 
research  team  have  completed  the  required  research  ethics  training.  You  are  required  to 
inform  this  office,  as  well  as  your  approving  IRB,  if  there  are  any  changes  or 
amendments  to  your  protocol.  No  further  review  or  approval  from  this  office  is  required. 
It  is  noted  that  the  NPS  IRB  approval  for  this  research  expires  5  March  2018.  A  copy  of 
the  NPS  IRB’s  approval  of  any  continuing  review  for  this  research  beyond  the  current 
expiration  date  must  be  submitted  to  this  office. 

6.  Per  reference  (f)  review  and  approval  by  the  USMC  Survey  Office  is  required  prior  to 
execution  of  this  survey. 

7.  If  you  have  any  questions  or  require  further  information,  please  don’t  hesitate  to 
contact  me  at  (703)  432-2566,  email  leah.watson@usmc.mil.  I  wish  you  success  with 
your  study  and  appreciate  your  patience  in  complying  with  our  review  and  approval 
process. 


L.  B.  WATSON 

Chair,  Institutional  Review  Board 


Copy  to: 

USMC  Survey  Office 
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4. 


USMC  SURVEY  OFFICE 


On  Apr  17,  2017,  at  11:47  AM,  Hogan  Maj  Kerry  A 
<kerrv.hoqan@usmc.mil>  wrote: 

Good  Afternoon, 

The  DC,  M&RA  has  approved  your  survey.  Your  SCN  is:  USMC-SR- 
17008. 

You  are  cleared  to  launch  the  survey. 

R/S 

Maj  Hogan 
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